Systems and Methods for Integrity Analysis of Clinical Data

ABSTRACT

Disclosed are implementations that include a method for dental data analysis including obtaining dental data for an individual, with the dental data being input radiographic image data of at least one dental object, and treatment data representative of one or more treatment procedures associated with the dental object, analyzing, by machine learning models, the input radiographic image data to identify one or more dental features associated with the dental object, and deriving, based on the treatment data and the identified one or more dental features, one or more integrity scores for the image data and the treatment data, with the one or more integrity scores being representative of potential integrity problems associated with the input radiographic image data and the treatment data. Deriving the integrity scores includes deriving a provider score representative of a potential integrity problem associated with a dental-care provider submitting the treatment data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part (CIP) application of, andclaims priority to, U.S. Non-Provisional application Ser. No.16/752,362, entitled “ESTIMATING MEASUREMENTS OF CRANIOFACIAL STRUCTURESIN DENTAL RADIOGRAPHS” and filed Jan. 24, 2020, and further claims thebenefit of, and priority to, U.S. Provisional Application No.63/050,333, entitled “MULTISYSTEM MACHINE LEARNING FOR UTILIZATIONINTEGRITY DETECTION” and filed Jul. 10, 2020, the contents of all ofwhich are herein incorporated by reference in their entireties.

BACKGROUND

Accurately measuring oral structures present on a two-dimensional dentalradiograph using currently available image processing methods isproblematic. During the process of taking a dental radiograph, an x-raysensor is positioned in a patient's mouth and an x-ray source is alignedwith the sensor outside of the patient's mouth. The term craniofacialstructure refers generally to the bones of the skull and face. The term“oral structures” (or “dental structures”) refers generally to naturalteeth, restorations, implants, and any other structure that relates tocraniofacial structures. Measurements of oral structures will typicallyinclude the relationship of the oral structure and craniofacialstructure. When the x-ray source is activated, x-rays are sent towardthe sensor, and any oral structures between the source and the sensorinfluence the resulting image. When an object is positioned upright anddirectly in the x-ray source's path, the object will be seen on thefinal image with minimal distortion. However, if the object's spatialorientation is changed in relation to the source and the sensor, theimage of the object will be distorted. In controlled systems, the degreeof distortion can be calculated if all angulations of the source,sensor, and object are known. In dentistry, however, the position of apatient's oral structures in relation to the sensor and the source canonly be estimated.

To address these issues, external calibration objects have been imagedalong with structures of interest to allow for image calibration. Thistype of calibration is only possible if the calibration object ispresent at the time that the radiographic image is captured.

Dental radiography is also finding increased use in the area of dentalinsurance underwriting and fraud detection. Dental insurance payers andcare providers deploy multiple, discrete programs within theadministrative systems to prevent unnecessary disbursement of clinicalservices and payment. These discrete programs include Fraud, Waste andAbuse (FWA) programs, clinical utilization review programs, andrule-based claim adjudication engines that check for coordination ofbenefits and frequency of limitation on payments.

Identifying and preventing FWA is one of the key functions in thebenefit payment cycle by dental insurance providers (payers) and largedentals service organizations (DSO) to ensure quality of care. FWAprogram is a mandatory program for administering and deliveringgovernment sponsored health and dental services in the United States. Inthe current provider-payer payment systems, FWA is commonly identifiedafter paying out a benefit through the claim adjudication process. FWAis detected using advanced analytics through execution of statisticaloutlier reports on historic data on provider utilization and payment.Fraud is verified by a special investigation unit (SIU) team, activelycollecting samples of data and images from clinical practices andcomparing the occurrences of fraud based on evidence from radiographicimages, chart notes and clinical utilization. This two-step process ofdetection and verification of FWA is a time consuming, resourceintensive and reactive process involving multiple disciplines that spanacross compliance, legal, clinical, and business.

Clinical utilization review program is a process of reviewing medicalnecessity for clinical treatments and appropriate payment reimbursementsin the provider-payer claim cycle. The selection of providers underclinical review is typically limited to standard clinical proceduresthat are costly and commonly upcoded or unbundled for submitting ahigher-paying service for reimbursement. Execution of a clinical reviewprogram requires high skills of clinical expertise with ability to readand interpret radiographs against submitted clinical services.

SUMMARY

The present disclosure relates generally to dental information analysis,including analysis of dental clinical data (also referred to as clinicaldata) to make a quick (expedited) determination of the reasonableness ofa proposed treatment plan submitted by dental-care providers (e.g., aspart of a request for treatment pre-approval or as a claim followingcompletion of a dental service). Implementations of dental informationanalysis described herein also include a comprehensive framework toanalyze clinical data and determine the accuracy and/or veracity(integrity level) of the information submitted (for proposed treatmentplans and/or claims for completed dental work). The various frameworksdescribed herein may be used in conjunction with image data calibrationand standardization approaches, including approaches for estimatingmeasurements of craniofacial structures by analyzing radiographic imagedata from different imaging modalities.

The dental data analysis solutions discussed below include a multisystemmachine learning for clinical data integrity detection. The proposedapproaches and solutions described herein include a combined, Fraud,Waste, and Abuse (FWA) detection methodology, and a continuous, on-goingclinical review framework that leverages advanced machine learningmodels deployed across multiple systems in the payment cycle. Thisproactive process can detect and prevent payments on duplicate services,same submission of images for multiple patients, and can cross-checkservices with objective evidence of clinical measurements from currentand historic radiography and provider chart notes. The embodimentsdescribed herein implement machine learning models that include aduplicate detector, image tampering detection, a phantom detector toidentify treatments and/or teeth that are not present, and/orinconsistencies in dental history (as it pertains to one or more teeth).

Another approach to facilitate an expedient and reliable analysis, andwhich can provide dental-care providers with a quick decision regardinga request submitted for an authorization of dental care plan for apatient, is based on determining metrics according to featuresidentifiable from supporting data submitted by the dental-careproviders, and assessing the reasonableness of the proposed treatmentplans according to the computed metrics. For example, when requestingapproval for a treatment plan for a patient, e.g., fitting a crown on adamaged (and potentially vulnerable) tooth, the provider will typicallysubmit radiographic images in support of the treatment plan. Theapproaches described herein are configured to identify (e.g., via atrained learning machine) various features discernable from the imagedata, based on which one or more metrics are derived. One such metricmay be a ratio of the size (e.g., area, length) of a damaged (ordiseased) portion of a tooth relative to another portion that isrepresentative of the overall size of the visible tooth structure (forexample the tooth crown area). Once the metric is derived it can beprocessed by rule-based algorithms and processes to determine if aprosthetic crown fitting is appropriate given the metric computed.

As will also be discussed in greater detail below, this approach too mayrequire an initial calibration procedure to generate calibrated imagedata that can be more reliably processed by machine learning enginesthat may have been trained (e.g., to identify various dentalfeatures/structures appearing in in the radiographic image datasubmitted by the provider) according to calibrated (standardized) imagesthat share a common scale and/or viewing perspective. Such calibrationcan reduce the occurrence of error.

Embodiments described herein may include calibration operations tocalibrate source image measurement data to measure the precise distancesand sizes on an image. For example, the width or height of a tooth inmillimeters or the depth of a decay and so on. The calibrationoperations seek to convert the pixels in a radiographic image to astandard unit (e.g., pixel-to-millimeter conversion that indicates thewidth and height of a pixel in millimeters). Thanks to this conversion,regardless of the image quality, actual distances can be accuratelyestimated (i.e., distances in millimeters between two point in an imageremains the same even if the resolution is changed by resizing)

The calibration solutions described herein achieve the calibration ofimage data without requiring the use of non-dental calibration objects(acting as landmark with known dimensionality), and instead rely on datathat may be available such as the sensor brand/model capturing theradiographic image. When the sensor information is not available otherintrinsic characteristics of the radiographic images may be used topredict the sensor information. Such data can be used to derive, amongother things, viewing angles, scales corresponding to the image data,and positioning information, with such derived data being used tocalibrate and/or correct alignment of the image data to increase resultaccuracy and confidence levels.

Advantages of the embodiments described herein include: 1) generatingprecise outputs such as area ratios for crown decision, distance (basedin part to the calibration procedures), or ratios (cej-bp and cej-apex)for scaling and root planing, 2) deriving decision outcomes (accept,deny, review, downcode, request for more information), 3) handlingmultiple image and document formats, performing image redaction,removing PHI, 4) generating predictions for intraoral images, 5)generating treatment quality scores (i.e., how well is a treatmentperformed), 6) performing image quality determination, performing imageenhancement, 7) computing prediction trustworthiness/uncertainty (e.g.,output such as “good,” “okay,” “bad” predictions), and 8) derivingprovider scores for individual providers, based on duplicate score,manipulation score, tooth/treatment presence score, ratio of quadrantsper claim for SRP, etc.

Thus, in some variations, a method for clinical data analysis isprovided that includes obtaining dental data for an individual, with thedental data including input radiographic image data for at least onedental object, and identifying, by one or more machine learning models,at least one first dental feature in the input radiographic image datafor the at least one dental object, and at least one other feature inthe dental object comprising at least partly a healthy dental structure(e.g., a portion of a tooth that is not diseased, restored, or damaged).The method additionally includes computing at least one dimensionedproperty representative of physical dimensions of the at least one firstdental feature and the at least one other feature comprising at leastpartly the healthy dental structure, deriving based on the at least onedimensioned property at least one dimensioned property ratio indicativeof an extent of a dental clinical condition associated with theidentified at least one first dental feature of the at least one dentalobject, and determining a treatment plan based on a comparison of thederived at least one dimensioned property ratio to a respective at leastone pre-determined threshold value.

Embodiments of the method may include at least some of the featuresdescribed in the present disclosure, including one or more of thefollowing features.

Computing at least one dimensioned property may include computing one ormore of, for example, areas of the identified at least one first dentalfeature and the at least one other feature, and/or lengths of theidentified at least one first dental feature and the at least one otherfeature. Deriving the at least one dimensioned property ratio mayinclude one or more of, for example, deriving an area ratio of an areafor the at least one first dental feature and area for the at least oneother feature, or deriving a length ratio of a length of the at leastone first dental feature and a length of the at least one other feature.

The at least one first dental feature may include one or more of, forexample, a decaying tooth portion for a tooth, a filling region for thetooth, a restoration, and/or bone loss. The at least one other featuremay include a clinical crown structure for the tooth.

Identifying, by the learning machine, the at least one first dentalfeature and the at least one other feature may include generating masks,by the learning machine, representative of the at least one first dentalfeature and the at least one other feature comprising at least partlythe healthy dental structure.

Determining the treatment plan may include one or more of, for example,automatically determining by machine learning model a proposed treatmentplan to treat an abnormal dental feature and/or determining based on thederived dimensioned property ratio whether to approve adental-care-provider treatment plan submitted by a dental-care provider.

Obtaining the dental data may include receiving source radiographicimage data represented according to pixel-based dimensions, andcalibrating the source radiographic image data to produce the inputradiographic image data represented in terms of estimated standard-unitdimensions, with the source radiographic image data being free of anynon-dental calibration objects.

The estimated standard-unit dimensions may include millimeter (mm)units.

Calibrating the source radiographic image data may include selecting asegmenter and/or an object detector, predicting source masks and sourcepoints (and/or keypoints) of the at least one dental object appearing inthe source radiographic image data using the segmenter and the objectdetector, providing the source radiographic image data and the imagemetadata, comprising the source masks and source points, to acalibration process selector, selecting by the calibration processselector at least one measurement process from a set of measurementprocesses according to the source radiographic image data and the imagemetadata, deriving a sensor pixel-to-standard-unit ratio using theselected at least one measurement process, and generating the inputradiographic image data and resultant calibrated metadata, comprisingcalibrated masks and points on the dental object, using calibratedmeasurements of the at least one dental object based on the sensorpixel-to-standard-unit ratio and the image metadata.

Deriving the sensor pixel-to-standard-unit ratio using the selected atleast one measurement process may include determining a sensor type forthe source radiographic image data, determining sensor characteristicsbased on the determined sensor type, determining pixel dimensions forthe source radiographic image data, and deriving the sensorpixel-to-standard-unit ratio based on the determined sensorcharacteristics and the determined pixel dimensions for the sourceradiographic image data.

Deriving the sensor pixel-to-standard-unit ratio using the selected atleast one measurement process may include identifying from the sourceradiographic image data teeth without restorations, determiningdistances in pixels between mesial and distal Cemento Enamel Junction(CEJ) points for the identified teeth, deriving a plurality ofpixel-to-standard-unit ratios using the determined distances in pixelsand based on pre-determined standard average distances between themesial and distal CEJ points for each of the identified teeth, andcomputing an average pixel-to-standard-unit ratio from the derivedplurality of pixel-to-standard-unit ratios.

Deriving a sensor pixel-to-standard-unit ratio using the selected atleast one measurement process may include determining one or more outerborders for respective one or more dental objects appearing in thesource radiographic image data, comparing the one or more outer bordersto 2D projections in a projection dictionary, the 2D projections beingat incremental distance and angles generated from 3D dental image data,to identify a match between the one or more outer borders and the 2Dprojections in the projection dictionary, estimating a viewing angle atwhich the source radiographic image data was obtained based on theidentified match between the one or more outer borders and the 2Dprojections in the projection dictionary, and deriving the sensorpixel-to-standard-unit ratio based on the estimated angle at which thesource radiographic image data was obtained.

Deriving a sensor pixel-to-standard-unit ratio using the selected atleast one measurement process may include detecting an implant structureappearing in the source radiographic image data, determining implantattributes based on the source radiographic image data for the detectedimplant structure, comparing the determined implant attributes for thedetected implant structure to stored implant attributes included inimplant data records, maintained in an implant structure database, forknown manufactured implants to identify a match between the determinedimplant attributes and the stored implant attributes included in thestored implant data records, and deriving the sensorpixel-to-standard-unit ratio based on stored geometrical informationassociated with a selected one of the implant data records that mostclosely matches the implant attributes determined from the sourceradiographic image data.

Deriving a sensor pixel-to-standard-unit ratio using the selected atleast one measurement process may include detecting an implant structureappearing in the source radiographic image data, determining implantattributes, based on the source radiographic image data for the detectedimplant structure, comparing the determined implant attributes for thedetected implant structure to stored implant attributes included inimplant data records, maintained in an implant structure database, forknown manufactured implants to identify a match between the determinedimplant attributes and the stored implant attributes included in thestored implant data records, determining an outer border for thedetected implant structure appearing in the source radiographic imagedata, comparing the outer border to 2D projections maintained in aprojection dictionary, the 2D projections being at incremental distanceand angles generated from 3D dental image data, to identify a matchbetween the outer border and the 2D projections in the projectiondictionary, estimating a viewing angle at which the source radiographicimage data was obtained based on the identified match between the outerborder and the 2D projections in the projection dictionary, and derivingthe sensor pixel-to-standard-unit ratio based on the estimated angle atwhich the source radiographic image data was obtained, and based onstored geometrical information associated with a selected one of theimplant data records that most closely matches the implant attributesdetermined from the source radiographic image data.

Deriving the sensor pixel-to-standard-unit ratio may include estimatingviewing angles for other dental objects detected in the sourceradiographic image data based on a position of the implant structurerelative to the other dental structure and based on the viewing angle atwhich the source radiographic image data was obtained, and deriving thesensor pixel-to-standard-unit ratio based on the estimated viewingangles for the other dental structures detected in the sourceradiographic image data, and based on the stored geometrical informationassociated with the selected one of the implant data records that mostclosely matches the implant attributes determined from the sourceradiographic image data.

In some variations, a system for clinical data analysis is provided thatincludes a communication interface to obtain dental data for anindividual, wherein the dental data comprises input radiographic imagedata for at least one dental object, one or more memory devices, and oneor more processor-based devices, coupled to the communication interfaceand to the one or more memory devices. The one or more processor-baseddevices are configured to identify, by one or more machine learningmodels, at least one first dental feature in the input radiographicimage data for the at least one dental object, and at least one otherfeature in the dental object comprising at least partly a healthy dentalstructure, compute at least one dimensioned property representative ofphysical dimensions of the at least one first dental feature and the atleast one other feature comprising at least partly the healthy dentalstructure, derive based on the at least one dimensioned property atleast one dimensioned property ratio indicative of an extent of a dentalclinical condition associated with the identified at least one firstdental feature of the at least one dental object, and determine atreatment plan based on a comparison of the derived at least onedimensioned property ratio to a respective at least one pre-determinedthreshold value.

In some variations, a non-transitory computer readable media isprovided, storing a set of instructions, executable on at least oneprogrammable device, to obtain dental data for an individual, whereinthe dental data comprises input radiographic image data for at least onedental object, and identify, by one or more machine learning models, atleast one first dental feature in the input radiographic image data forthe at least one dental object, and at least one other feature in thedental object comprising at least partly a healthy dental structure. Thecomputer readable media include further instructions to compute at leastone dimensioned property representative of physical dimensions of the atleast one first dental feature and the at least one other featurecomprising at least partly the healthy dental structure, derive based onthe at least one dimensioned property at least one dimensioned propertyratio indicative of an extent of a dental clinical condition associatedwith the identified at least one first dental feature of the at leastone dental object, and determine a treatment plan based on a comparisonof the derived at least one dimensioned property ratio to a respectiveat least one pre-determined threshold value.

Embodiments of the system and the computer-readable media may include atleast some of the features described in the present disclosure,including at least some of the features described above in relation tothe method.

In some variations, another method for dental data analysis is providedthat includes obtaining by a computing system dental data for anindividual, the dental data comprising input radiographic image data ofat least one dental object, and treatment data representative of one ormore treatment procedures associated with the at least one dentalobject, analyzing, by one or more machine learning models implemented bythe computing system, the input radiographic image data to identify oneor more dental features associated with the at least one dental object,and deriving, by the computing system, based on the treatment data andthe identified one or more dental features associated with the at leastone dental object, one or more integrity scores for the inputradiographic image data and the treatment data, with the one or moreintegrity scores being representative of potential integrity problemsassociated with the input radiographic image data and the treatmentdata. Deriving the one or more integrity scores includes deriving aprovider score representative of a potential integrity problemassociated with a dental-care provider submitting the treatment data.

Deriving the provider score may include computing an outlier scorerepresentative of a level of deviation between a treatment plan,specified in the treatment data for the at least one dental object toremedy a dental condition identified in the treatment data for theindividual, and treatment plans to treat similar dental conditionsassociated with archived treatment data and archived radiographic imagedata for a plurality of other individuals.

Deriving the provider score may include determining, by the computingsystem, one or more possible treatment plans to remedy a dentalcondition identified in the treatment data for the individual, andcomputing an aggressiveness score representative of a complexity leveldifference between a specified treatment plan submitted by thedental-care provider to remedy the dental condition and the determinedone or more possible treatment plans.

Deriving the provider score may include computing a phantom diseasescore representative of a level of consistency between a treatment planspecified in the treatment data for the at least one dental object toremedy a dental condition identified in the treatment data for theindividual, and identified features of the input radiographic image datadetected by the computing system.

Computing the phantom disease score may include one or more of, forexample, performing image manipulation detection on the inputradiographic image data to determine whether a portion of the inputradiographic image data was modified, and/or determining, based onfuture image data, that the treatment was never performed.

Computing the phantom disease score performing a duplicate ornear-duplicate image detection on the input radiographic image data todetermine whether a portion of the input radiographic image data,relating to identified dental condition for the at least one dentalobject, fully or substantially matches a portion of a previously storedradiographic image.

Deriving the provider score may include computing, based on thetreatment data associated with the input radiographic image data for theindividual, and based on archived treatment data for one or moreindividuals treated by the dental-care provider, a phantom treatmentscore representative of extent to which the dental-care provider submitstreatment plans inconsistent with associated dental conditionsidentified form the treatment data for the individual and from thearchived treatment data.

Deriving the one or more integrity scores may include identifying, by atleast one of the one or more machine learning models, at least one firstdental feature in the input radiographic image data for the at least onedental object, and at least one other feature in the dental objectcomprising at least partly a healthy dental structure, computing atleast one dimensioned property representative of physical dimensions ofthe at least one first dental feature and the at least one other featurecomprising at least partly the healthy dental structure, and derivingbased on the at least one dimensioned property at least one dimensionedproperty ratio indicative of an extent of a dental clinical conditionassociated with the identified at least one dental feature of the atleast one dental object. The dimension property ratio(s) can be used(e.g., using rules that are based on comparisons to respectivepre-determined threshold/reference values) a treatment plan (e.g.,either an actual recommended treatment plan, or a decision about whetherto accept or reject, or take some other action, with respect to aproposed treatment plan).

Analyzing the input radiographic image data may include detectinganomalous features in the input radiographic image data, includingdetermining one or more of, for example, whether a portion of the inputradiographic image data substantially matches a portion of a previouslystored radiographic image, and/or whether a portion of the inputradiographic image data was modified.

In some variations, another system for dental data analysis is providedthat includes a communication interface to obtain dental data for anindividual, the dental data comprising input radiographic image data ofat least one dental object, and treatment data representative of one ormore treatment procedures associated with the at least one dentalobject, one or more memory devices, and one or more processor-baseddevices, coupled to the communication interface and to the one or morememory devices. The one or more processor-based devices are configuredto analyze, by one or more machine learning models implemented by thecomputing system, the input radiographic image data to identify one ormore dental features associated with the at least one dental object, andderive, by the computing system, based on the treatment data and theidentified one or more dental features associated with the at least onedental object, one or more integrity scores for the input radiographicimage data and the treatment data, with the one or more integrity scoresbeing representative of potential integrity problems associated with theinput radiographic image data and the treatment data. The one or moreprocessor-based devices configured to derive the one or more integrityscores are configured to derive a provider score representative of apotential integrity problem associated with a dental-care providersubmitting the treatment data.

In some variations, another non-transitory computer readable media isprovided that stores a set of instructions, executable on at least oneprogrammable device, to obtain by a computing system dental data for anindividual, the dental data comprising input radiographic image data ofat least one dental object, and treatment data representative of one ormore treatment procedures associated with the at least one dentalobject, analyze, by one or more machine learning models implemented bythe computing system, the input radiographic image data to identify oneor more dental features associated with the at least one dental object,and derive, by the computing system, based on the treatment data and theidentified one or more dental features associated with the at least onedental object, one or more integrity scores for the input radiographicimage data and the treatment data, with the one or more integrity scoresbeing representative of potential integrity problems associated with theinput radiographic image data and the treatment data. The instructionsto derive the one or more integrity scores include one or moreinstructions to derive a provider score representative of a potentialintegrity problem associated with a dental-care provider submitting thetreatment data.

Embodiments of the other system and the other computer-readable mediamay include at least some of the features described in the presentdisclosure, including at least some of the features described above inrelation to the methods, the first system, and the firstcomputer-readable media.

Any of the above variations of the methods, systems, and/orcomputer-readable media, may be combined with any of the features of anyother of the variations of the methods, systems, and computer-readablemedia described herein.

Other features and advantages of the invention are apparent from thefollowing description, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects will now be described in detail with referenceto the following drawings.

FIGS. 1A-1B depict example systems for performing calibration and dataanalysis processes.

FIG. 2 is a flow diagram depicting an example method for an imageprocessor for determining optionally image orientation and image type,predicting masks and points on images, and providing image and imagemetadata.

FIGS. 3A-3B are examples of radiographic image types detected by theimage processor.

FIGS. 4A-4E are examples of predicted masks and points that may beprovided by the image processor.

FIG. 5 is a flow diagram depicting a measurement processor thatdetermines a sensor pixel to mm (millimeter) ratio by selecting from atleast one measurement method, and provides a calibrated measurement anda threshold.

FIG. 6 is a flow diagram depicting a first measurement methodimplemented by the measurement processor of FIG. 5 for obtaining sensorpixel to mm ratio when the sensor can be determined.

FIG. 7 is a flow diagram depicting a second example method implementedby the measurement processor of FIG. 5 that determines a sensor pixel tomm (millimeter) ratio using radiographic images of natural teeth when animage sensor cannot be determined.

FIGS. 8A-8B are examples of measurements performed on radiographicimages of natural teeth according to the method of FIG. 7.

FIG. 9 is a flow diagram depicting a third example method implemented bythe measurement processor of FIG. 5 that determines a sensor pixel to mm(millimeter) ratio using radiographic images of natural teeth.

FIG. 10 illustrates the projection of a 3D (three-dimensional) surfaceonto a plane at incremental angles to generate a dictionary ofprojections.

FIG. 11 is a flow diagram depicting a fourth example method implementedby the measurement processor of FIG. 5 that determines a sensor pixel tomm (millimeter) ratio by identifying an implant from an implantdatabase.

FIG. 12 is a flow diagram depicting a fifth example method in themeasurement processor of FIG. 5 that determines a sensor pixel to mm(millimeter) ratio using the metadata from the third and fourth methods.

FIG. 13 is an example graphical user interface (GUI) screen forpresenting desired calibrated measurements related to a radiographimage.

FIG. 14 is a block diagram illustrating an example softwarearchitecture, various portions of which may be used in conjunction withvarious hardware architectures herein described.

FIG. 15 is a block diagram illustrating components of an example machineconfigured to read instructions from a machine-readable medium andperform any of the features described herein.

FIG. 16 is a block diagram of an example system to perform clinical dataanalysis.

FIG. 17 is an example dashboard displaying a radiographic image on whichresultant output markings corresponding to detected dental features havebeen superimposed.

FIG. 18 is a flowchart of an example procedure for clinical dataanalysis.

FIG. 19 is a flow diagram for multisystem machine learning for clinicaldata integrity analysis.

FIG. 20 is a flow diagram for analyzing radiographic data.

FIG. 21 is a flow diagram for a provider score determination process.

FIG. 22 illustrates data structures accessed in the course of performinga phantom disease review.

FIG. 23A shows a manipulated x-ray image.

FIG. 23B shows duplicate x-ray image submissions.

FIG. 24 is a flowchart of an example procedure for clinical dataintegrity detection.

FIG. 25 is a system diagram for utilization analysis.

FIGS. 26A-B include examples of potential tampered dental radiographicimages.

Like reference symbols in the various drawings indicate like elements.

DESCRIPTION

The present disclosure discusses various techniques and approaches foranalyzing data submitted, for example, by dental care providers, inorder to assess the veracity of such submitted data in order to identifyerroneous or suspicious data, and to assess in an expedient and reliablemanner, the reasonableness/appropriateness of treatments plans submittedby the dental care providers (i.e., assess their medical necessity).Data submitted by providers may include errors (which may have resultedfrom innocent oversights, due to the indication of improper codes, orthe inclusion of incorrect supporting data), or may be indicative ofattempts to either recover money for fraudulent treatment plans, orattempts to bill (or receive pre-authorization) for improper orunnecessary treatment plans. The analysis performed by theimplementations described herein is generally performed by processing,using machine learning engines, clinical/treatment data (also referredto as clinical data) from providers (which includes descriptive data,possibly provided using natural language descriptions, or charts or bystandardized codes and pre-determined text-based descriptions that areavailable in lists/libraries of descriptive data) and supporting dentalimage data (in the form of dental photographs and radiographic imagedata).

As will be discussed in greater detail below, the implementationsdescribed herein are configured to handle input data from numerousdifferent sources (e.g., different dental-care providers, submittingclaims and pre-authorization requests to different insurance companiesusing different claim processing and review procedures). In mostsituations, the standard-unit distances may not be provided by theprovider, and consequently the payers (such as insurance companies) canonly crudely estimate the standard-unit distances within the x-rays(eyeballing them). To quantify the measurements on a radiographic image,the standard-unit dimensions of it need to be determined. In order toensure that the learning machine models implemented for the variousengines can provide consistent and accurate output data no matter thesubmitting source (where each source, namely each individual dentalprovider, may provide image data captured using different equipment, andunder different image-viewing conditions), a calibration procedure maybe used, at least in some embodiments, to measure image data that has asubstantially uniform scale, and/or may have a substantially uniformviewing perspective. For this to measurement, calibration is based onother sources of information, which may include previously captureddental image data for a particular patient for whom a claim or apre-authorization request is being submitted, archives of image data formultiple patients (which may be grouped according to the dental-careproviders treating them), data repositories of technical and performancespecifications of the imaging equipment (sensors) being used to captureimage data, etc.

As will further be discussed in greater detail below, the presentdisclosure describes approaches for performing a quick (expedited)clinical data analysis (and, in some embodiments, providesresponses/answers within a few seconds or a few minutes from receipt ofrequest data) that assesses whether treatment plans submitted bydental-care providers are reasonable and/or proper in view of thesupporting materials accompanying the requests (e.g., the so-calledclinical data). In some examples, the analysis may be performed byderiving metrics based on computation of size (e.g., area) of dentalfeatures identified (e.g., by a machine learning engine) in dentalimages. An example of a dental metric that can be used is an area ratiovalue computed as a ratio of the area of a diseased (damaged) portion ofa tooth (that is to be treated) relative to the overall size (e.g.,area) of the tooth (as may be represented by the clinical crown portionconstituting the tooth structure). It is to be noted that because such ametric is a ratio (and thus is a scalar value that does not necessarilydepend on absolute size of the measured features), this clinical dataanalysis could be completed without necessarily needing to firstcalibrate (or standardize) the input image data (this can reduce thecomputational effort and yield results more quickly than in thesituation where calibration is first performed).

The present disclosure also discusses a comprehensive framework fordetermining data integrity of dental data submitted by providers, so asto detect oversight errors from the submitting sources, detect potentialattempts to bill for medically unnecessary dental procedures, and detectoutright attempts at fraud (by submitting manipulated/doctored images,or by submitting images obtained from different patients, etc.) Becausethis framework, as will be discussed in greater detail below, leveragesarchived image data for the patient for whom a claim or pre-approvalrequest is being submitted, and multitude of other patients, theframework may, in some examples, require calibration of radiographicimage data (according to, for examples, the techniques and approachesdescribed herein, i.e., without requiring that calibration objects beused.

The present disclosure is organized as follows. First, the particularsof the proposed calibration procedures (which may be used in conjunctionwith the data processing and analysis frameworks) will be described.Next, a proposed approach for analyzing clinical data according tometrics derived from the areas of features identified in theradiographic images will be described. Lastly, a comprehensive frameworkfor utilization integrity and fraud detection (leveraging archivalpatients' dental data, and data regarding dental providers' historicalbehavior vis-à-vis submission of claims) is described.

Dental Image Data Calibration and Measurement Estimation

The present disclosure describes calibrations/standardizationimplementations that are free of foreign (non-dental) calibrationobjects, and are configured to estimate measurements (resulting incalibrated/standardized image data) using, for example, a patient'sunique anatomical oral structures as calibration objects without theneed for an external calibration object. Additionally, the proposedapproaches can calculate the angle between a patient's oral structuresand the x-ray source and sensor, when 3-dimensional measurements ofstructures featured in the produced radiographic images are available.

Benefits and advantages provided by such implementations can include,but are not limited to, a technical solution to the technical problemsof using a patient's unique anatomical oral structures as calibrationobjects to make calibrated measurements without the need for an externalcalibration object. Technical solutions and implementations providedherein optimize the process of obtaining calibrated measurements of oralstructures featured in 2D dental radiographs. The benefits andadvantages provided by these technology-based solutions yield moreuser-friendly applications, increased accuracy and increased system anduser efficiency.

The methods, systems, and other implementations described herein mayinclude, or otherwise make use of, a trained machine-learning model toidentify contents related to input data. Machine learning (ML) generallyinvolves various models, algorithms, and processes that allow acomputing machine to automatically learn/adapt over time to optimize itsperformance. The machine learning approaches can be based onoptimization processes that can be employed to predict events, classifyentities, diagnose problems, and model function approximations. As anexample, a system can be trained using data generated by a ML model inorder to identify patterns in dental radiographs. Such determinationsmay be made following the accumulation, review, and/or analysis of userdata from a large number of users over time as well as individualpatient data, that may be configured to provide the proposed MLapproaches with an initial or ongoing training set. In addition, in someimplementations, supplemental training data may be intermittentlyprovided to fine-tune or increase the effectiveness of the machinelearning model implemented by the machine learning system.

In different implementations, a training system may be used thatincludes an initial ML model (which may be referred to as an “ML modeltrainer”) configured to generate a subsequent trained ML model fromtraining data obtained from a training data repository or fromdevice-generated data. The generation of this ML model may be referredto as “training” or “learning.” The training system may include and/orhave access to substantial computation resources for training, such as acloud, including computer server systems adapted for machine learningtraining. In some implementations, the ML model trainer is configured toautomatically generate multiple different ML models from the same orsimilar training data for comparison. For example, different underlyingML algorithms may be trained, such as, but not limited to, decisiontrees, random decision forests, neural networks, deep learning (forexample, convolutional neural networks), support vector machines,regression (for example, support vector regression, Bayesian linearregression, or Gaussian process regression). As another example, size orcomplexity of a model may be varied between different ML models, such asa maximum depth for decision trees, or a number and/or size of hiddenlayers in a convolutional neural network. As another example, differenttraining approaches may be used for training different ML models, suchas, but not limited to, selection of training, validation, and test setsof training data, ordering and/or weighting of training data items, ornumbers of training iterations. One or more of the resulting multipletrained ML models may be selected based on factors such as, but notlimited to, accuracy, computational efficiency, and/or power efficiency.In some implementations, a single trained ML model may be produced. Itis to be noted that in some situations, processes such as opticalcharacter recognition (OCR) may be used in combination with ML models.For example, OCR may be used for extracting text from images and thenusing it in conjunction with ML models (e.g., to identify/detect toothclinical crowns). In some embodiments, the ML training system may alsobe configured to generate training data using generative adversarialnetworks (GANs, as further discussed below in greater detail), improvedisplay of images on the front end, and/or improve processing of imagesfor subsequent models by eliminating noise.

The training data may be continually updated, and one or more of themodels used by the system can be revised or regenerated to reflect theupdates to the training data. Over time, the training system (whetherstored remotely, locally, or both) can be configured to receive andaccumulate more and more training data items, thereby increasing theamount and variety of training data available for ML model training,resulting in increased accuracy, effectiveness, and robustness oftrained ML models.

FIG. 1A illustrates an example system 100 upon which aspects of thisdisclosure may be implemented. The system 100 may include a computingplatform (e.g., a personal computer) 105 executing software thatimplements an image processor 200 (e.g., implemented as a ML model)which is coupled to receive a radiographic image 120 containing an oralstructure 125 that could comprise a natural tooth, an implant, arestoration, etc., that further relates to craniofacial structures suchas bones. The image processor 200 provides output, such as imagemetadata (e.g., detected features that appear in the image, and theircorresponding labels) to a measurement processor 300 which in turnprovides calibrated output (a calibrated measurement) based on the imagemetadata and data from a library 150 that may include an X-ray sensordatabase, an implant database, a population based anatomical averagesdatabase, and a patient specific database, as more particularlydescribed below.

A training mechanism 140 provides a machine-learning based trainingmechanism, as mentioned above, for training aspects of the imageprocessor 200 for generating (predicting), masks, labels, features andpoints of the oral structure 125 (e.g., identifying features withtext-based identifiers, marking certain features with outlines,representing features as geometric shapes, etc.) A display 160presents/renders a graphical user interface (GUI) for displaying thecalibrated measurement. The specific masks that the ML model implementsfor the image processor may depend on the specific measurementcalibration process that is applied to the image to determine the scalethat is to be used (e.g., determine the pixel-to-millimeter ratio forthe imager). It is to be noted that in some situations only bone levelsneed measurement in mm, whereas ratio-based calculations (including areacalculations), could be performed in the pixel space.

As will be discussed in greater detail below, those measurementprocesses include, for example, the known sensor measurement process,the oral structure measurement process, the 3D surface data measurementprocess, the known implant measurement process, and/or the implant anglemeasurement process.

In some embodiments, the training mechanism 140 (also referred to alearning engine controller/adapter) is configured to determine and/oradapt the parameters (e.g., neural network weights) of the learningengine that would produce output representative of masks andrepresentation of detected dental objects and features appearing in theimage data. To train the ML-based image processor 200, training datacomprising masks, labels, or other representations (collectivelyreferred to as training output data) is provided to the trainingmechanism 140. The training output data thus defines samples of theground truth that are used to train the ML-based image processor 200(offline and/or during runtime). This training data may be used todefine the parameter values (weights, represented as the vector θ)assigned to links of, for example, a neural network implementation ofthe machine learning engine. The weight values may be determined, forexample, according to a procedure minimizing a loss metric betweenpredictions made by the neural network in response to the underlyingimage data provided to the ML-engine of the image processor, and themasks, labels, or other output representations of the training data(e.g., using a stochastic gradient descent procedure to minimize theloss metric). The computed parameter values can then be stored at amemory storage device (not shown) coupled to the image processor 200and/or to the training mechanism 140. After a learning-engine basedimplementation of the image processor 200 has become operational(following the training stage) and can process actual runtime data,subsequent run-time training may be intermittently performed (at regularor irregular periods) to dynamically adapt the image processor to new,more recent training data samples, and to newer model architectures anddata label groupings, in order to maintain or even improve theperformance of the image processor 200.

FIG. 1B illustrates an alternative embodiment of the example system 100upon which aspects of this disclosure may be implemented as a webservice, including the computer 105 operating as a client device forexecuting client application 190 software that is coupled to receive theradiographic image 120 containing the oral structure 125 that couldcomprise a natural tooth, an implant, a restoration, etc. A server 170runs a server-side application such as in a cloud service executingsoftware that communicates with the application 190 via a network 180 toreceive the radiographic image 120. Server 170 executes software forimplementing the image processor 200 that provides image metadata to themeasurement processor 300, which in turn provides a calibratedmeasurement based on the image metadata and data from the library 150that may include an X-ray sensor database, an implant database, apopulation based anatomical averages database, and a patient specificdatabase. The training mechanism 140 provides a machine-learning basedtraining mechanism for training aspects of the image processor forbetter predicting masks and points of the oral structure 125. Thedisplay 160, as part of the application 190, presents or renders thedisplay of a graphical user interface (GUI) for displaying thecalibrated measurement received from the server 170.

FIG. 2 depicts the operation of the image processor 200 that receivesthe radiographic image 120 containing oral structure 125. An initialtransformation step that converts images to a standard format such asJPEG or PNG may be required. At step 202, the image orientation isoptionally detected, if the image orientation is not correct, theorientation is optionally adjusted at step 204. Steps 202-206 mayoptionally be performed by the image processor 200. When processed byinsurance related data analysis system, the procedure depicted in FIG. 2may also include image extraction/cropping operations performed on imagedata attachments.

FIG. 3A depicts an example of the radiographic image 120 at a correctorientation and at a 90-degree rotation that would require adjustment.At step 206, an image type for the radiographic image 120 is optionallydetermined. FIG. 3B depicts the radiographic image 120 with exampleimage types that include bitewing and periapical, among others.

Referring back to FIG. 2, in step 208, masks and points on theradiographic image 120 are predicted by a segmenter and object detector.Depending on the image type as determined in step 206, appropriate pairsof segmenters and object detectors are chosen from a set of segmentersand object detectors 210-214, including selecting among BitewingSegmenter and Object Detector 210 (in response to the image type beingdetermined, at 206, to be a bitewing image type), Periapical Segmenterand Object Detector 212 (in response to the image type being determined,at 206, to be a periapical image type), and Panoramic Segmenter andObject Detector 214 (in response to the image type being determined, at206, to be a panoramic image type) in order to provide the desired masksand points prediction. The Bitewing Segmenter and Object Detector 210,Periapical Segmenter and Object Detector 212, and Panoramic Segmenterand Object Detector 214 may use different Deep Learning (DL) based MLmodels for prediction. For example, bitewing specialized anatomytraining models may be used to achieve better prediction (e.g., betterlabelling) of bitewing images. The selected DL model may predict masksfor many labels such as tooth number, general tooth area, bone, enamel,restorations such as crown, filling/inlay, onlay, bridge, implants etc.The DL model itself may be an amalgam of several detection architectures(implementing multiple DL models that are combined to provide meaningfulresults). The best model identified with the help of metrics such asIntersection over Union (IoU) and bone level (distance between CementoEnamel Junction (CEJ) and bone point) against a test set is chosen forthe particular label. Further, the model may also directly predict theCEJ, and alveolar bone crest (hereafter called bone) points per toothnumber. The model provides two ways of getting CEJ and bone points,which can be used to improve the confidence of the measurements. Themasks and points are both in terms of pixel measurements. It is to benoted that any number of different image types coupled with anappropriate segmenter and object detector may be employed as desirable.The training mechanism 140 provides for training the ML models for theBitewing Segmenter and Object Detector 210, the Periapical Segmenter andObject Detector 212, and the Panoramic Segmenter and Object Detector 214in order to provide the mask and points prediction. The ML models mayalios include semantic segmentation, instance segmentation, objectdetection, keypoint detection, etc.

FIG. 4A-4E depict various non-limiting examples of the masks and pointsthat may be handled by the Segmenter and Object Detectors 210-214 forthe oral structures 125 in the radiographic image 120, including theBone Mask, Tooth Number Identification, Tooth Mask, EnamelIdentification, and Restorations. Turning back to FIG. 2, in step 216,the radiographic image 120 and image metadata comprising the image typeand associated masks (as determined by the image processor 200) andpoints are provided to the measurement processor 300 for furtherprocessing that includes measurement estimation and calibration.

FIG. 5 depicts the flow diagram and operation of the measurementprocessor 300 shown in FIG. 1. As illustrated, the measurement processor300 receives the radiographic image 120 and the image metadata from theimage processor 200. The radiographic image 120 and image metadata(including masks, labels, segmented representations, etc.) providemeasurements in terms of pixels that require calibration via selectionof one or more of methods 304, 320, 350, 370, and 380 by a selector 302to obtain a sensor pixel to mm (millimeter) ratio 385 (it is to be notedthat other unit lengths may be used to establish other calibrationratios). Selector 302 may operate according to a set of rules thatdetermine which of the methods 304, 320, 350, 370, and 380 are selected.If the image metadata includes a known sensor type or a sensor type thatcan be determined, then the Known Sensor Measurement Method 304 isselected. If the image metadata does not include a known sensor type orthe sensor type cannot be determined, then the Oral StructureMeasurement Method 320 is selected. If 3D surface data measurement datais available, such as from a library 150 (depicted in FIG. 1, and whichis configured to store archived medical/dental data for the particularpatient associated with the radiographic image 120, and store archiveddental/medical data for other patients), then 3D Surface DataMeasurement Method 350 is selected. If the image metadata includes aknown implant as part of the image metadata, then the Known ImplantMeasurement Method 370 is selected. Further, if both 3D surface datameasurements are available along with known implant measurement datafrom methods 350 and 370, then Implant Angle Measurement Method 380 canalso be chosen to obtain the (generally) most accurate measurement.Based on radiographic image 120 and the image metadata from imageprocessor 200, in combination with calibration measurement 390, themeasurement confidence level 392 can be determined along with athreshold 394. The calibrated measurement 390 along with threshold 394can subsequently be provided to the display 160 as part of a userinterface, and communicated to downstream modules such as a utilizationintegrity detector and/or a fraud detector. The detailed operation ofmethods 304, 320, 350, 370 and 380 is explained in further detail belowin FIGS. 6-12.

An alternative embodiment of selector 302 includes a machine learningimplementation in which any of various combinations of the measurementmethods 304, 320, 350, 370, and 380 can be selected according tooptimization techniques that provide for calibrated measurement and maybe optimized for measurement speed, accuracy, or threshold as desiredusing ML as provided by the training mechanism 140. The machine learningmodel may be based on specific patient characteristics as well ascrowdsourced methods that provide for overall measurement techniques.

FIG. 6 depicts a flow diagram for the Known Sensor Measurement Method304 (performed in response to selection of that process by the selector302). In situations where the radiographic image metadata includes aknown sensor type or a sensor type can otherwise be determined frominformation contained in the library 150 (which may include known sensortypes), the length and width measurements of the input image aremeasured in pixels (in step 306). In step 308, the length and width arecompared to a database of sensors from the library 150 and from thatcomparison, the sensor is identified (at 312). Once the sensor has beenidentified, the sensor pixel to mm (millimeter) ratio for the sensor canbe provided (at 312) according to the database of sensors from thelibrary.

FIG. 7 depicts Oral Structure Measurement Method 320 as selected byselector 302. In response to a determination that the radiographic imagemetadata does not include a known sensor type or that the sensor typecannot be otherwise determined, Oral Structure Measurement Method 320 isselected as an alternative to obtain calibrated measurements using oralstructures identified in the radiographic image. Thus, in step 322,natural teeth without restorations are identified as oral structures(this identification may be made by the image processor 200 of FIG. 1A,as one of the label outputs provided by one of ML models implemented bythe image processor 200). In step 324, a distance (in pixels) ismeasured between the mesial and the distal CEJ points on the identifiedteeth. In step 326, selected conversions of pixels to millimeter (orsome other standard-unit of length) using a database of anatomicalaverages for each tooth, as obtained from library 150, are made. In step328, ratios of CEJ distances between all pairs of teeth to identifyoutliers are determined. In Step 330, if there is a single outlier, thenthat outlier may be eliminated. In step 332, if there are multipleoutliers, teeth with the least standard deviation from the CEJ distancesprovided in the database are selected. In step 334 an average pixel tomillimeter ratio of the remaining teeth is calculated. In step 336 apixel to millimeter ratio for the selected tooth is made.

The process of identifying outliers and eliminating them may be done inalternative ways, including, for example, using Kalman filtering, toeliminate sources of error to the extent possible. An alternativeembodiment for steps 330 through 334 may include the use machinelearning methods which provide a more flexible method of choosing whichtooth and which measurement to obtain the pixel to millimeter ratio. Forexample, decisions based on trained ML models may identify a particulartooth or a particular set of measurements that will likely provideoptimal results based on experience with a particular patient or aparticular set of patients to obtain the most reliable measurement inthe radiographic image 120.

FIGS. 8A and 8B illustrate the measurements performed in step 324 of theimage processor 200 operating on the radiographic image 120 and oralstructure 125 to identify the appropriate points necessary formeasurements of the oral structure 125. The outer border of the dentalcrowns is traced using at least one of the Segmenter and ObjectDetectors 210-212 (selected according to the image type). A minimumdistance of each pixel of the outer border of the occlusal surface fromthe CEJ to CEJ line is calculated and an occlusal surface line 344parallel to the CEJ to CEJ point line 340 is generated at the averageminimum distance of all pixels included within the occlusal surface. AnCEJ-occlusal plane line distance 346 between the occlusal surface line344 and the CEJ to CEJ line 340 is measured in pixels. A ratio betweenthe CEJ-occlusal plane line distance 346 and a CEJ-boney crest distance348 is calculated and compared with reference data from the library 150to determine if local bone loss is present above a determined bone lossthreshold. FIG. 9 depicts the 3D Surface Data Measurement Method 350performed when selected by the selector 302. Surface Data MeasurementMethod 350 may be used when sensor can or cannot be determined. If thesensor information is present the 3D surface data provides increasedaccuracy of measurement by accounting for image angulations. If thesensor information is not present, the 3D surface data may be used toprovide mm to pixel ratio (or to provide some otherpixel-to-standard-unit ratio).

Data from 3D dental imaging, such as an optical surface scan and/or conebeam computed tomography, is used to generate a dictionary of 2Dprojections of the oral structures projected onto a plane at incrementaldistances and from incremental angles. A computational minimizationproblem will be utilized to arrive at final solution. A 2D (twodimensional) radiographic image is analyzed (e.g., by the imageprocessor 200) to determine the outer borders of craniofacial structuresof interest within the image (the outer border can be generated as anoutput of a ML model trained to generated outer border of craniofacialstructures, or through some analytical/filter-based processing), and thelibrary of two-dimensional projections is searched to determine a matchbetween the radiographic image and a two-dimensional projection from thelibrary. The matched images can then be used to determine the angulationat which the original 2D radiographic image was taken. 3D structuresthat can be used to calculate angulation of the x-ray source comparedwith the imaged structures include dental implants, dental restorations,and dental hard tissue structures (e.g., teeth and bone). When theangulation of the imaged oral structure 125 is known, the totaldistortion of the image can be calculated, and the distances measured ondental radiographs can be calibrated.

FIG. 10 depicts a dictionary of projections 352 that can be generated byprojecting a 3D surface, in this case the oral structure 125 onto aplane at incremental angles as shown across angles A, B, and C. Anynumber of possible angles and projections may be chosen to create thedictionary of projections (at step 352 of FIG. 9) to obtain a desiredlevel of precision. The dictionary of projections may be stored in, andsubsequently retrieved from, the library 150. Potential applications forcalibrated distance measurements in dentistry may include, but are notlimited to: measuring root canal length during endodontic procedures,measuring root canal length and length of canal preparation duringdental post placement procedures, measuring vertical and horizontaldimension of bone in prospective implant placement sites, determiningvertical level of alveolar bone in relation to adjacent structures(e.g., teeth) to monitor periodontal disease progression, anddetermining distance between dental restorations and the underlyingalveolar bone, measuring distance between inferior alveolar nerve andadjacent structures, objects or osteotomy sites, measuring distancebetween maxillary sinus floor and adjacent structures, objects orosteotomy sites, and determining angulation of implants compared toadjacent teeth and structures.

Referring back to FIG. 9, in step 354, an outer border of a tooth,corresponding to the oral structure 125 in the radiographic image 120,are identified. In step 356, the outer border is compared with thedictionary of projections to determine the approximate angle at whichthe radiographic image 120 was taken. In step 358, projections may becreated or retrieved from the library 150 at increasingly finerincrements as desired to refine an angulation determination. In step360, measurements from the radiographic image 120 and oral structure 125are compared with true measurement values from the dictionary ofprojections 352 to calculate a distortion value related to theapproximate angle that can be used to determine the pixel to millimeterratio.

FIG. 11 depicts the Known-Implant-Measurement Method 370 performed whenselected by selector 302. In step 372, an area of the input radiographicimage 120 corresponding to a dental implant within the radiographicimage 120 image are identified (e.g., by the image processor,implementing an ML model for identifying and marking dental implants).In step 374, descriptive attributes of the dental implant are determined(e.g., by a same or different ML model implemented by the imageprocessing 200, and applied to the source radiographic image 120). Instep 376, the descriptive attributes of the dental implant are comparedto those within a dental implant database from library 150. In step 378,the output representing the implant(s) are analyzed to determinedescriptive attributes of the implant(s), including for example: type ofimplant interface, flange shape, presence or absence collar and shape ofcollar, presence or absence of micro threading, presence or absence ofimplant taper and location of taper, presence or absence of threads andnumber of threads, presence or absence of mid-body grooves, shape ofimplant apex, presence or absence of open implant apex, presence orabsence of holes and shape of holes present, presence or absence of anapical implant chamber, and presence or absence of apical grooves. Alist of identified attributes and identifies implants with identicalattributes stored within a database of previously classified implantmodels is retrieved (e.g., from the library 150). Information retrievedabout the implant may include the manufacturer, model name and/ornumber, years during which the reported implant model was produced bythe manufacturer (as obtained from the implant database), etc. Given thedetailed geometric information available about the dental implant, thepixel to millimeter ratio 385 can then be determined.

FIG. 12 depicts the Implant-Angle-Measurement-Method 380, when selectedby the selector 302. This methodology is a combination of the 3D SurfaceData Measurement Method 350 described above in relation to FIG. 9, andthe Known Implant Measurement Method 370 described in relation to FIG.11. Thus, under this combined methodology, the Implant Angle MeasurementMethod 380 may be performed when the combination of both methods 350 and370 have been previously selected by the selector 302. In this way, themost accurate measurement data can be derived based on the outputs ofmethods 350 and 370 to be further processed by method 380.

For a radiographic image 120 featuring an identified dental implant, animplant database with associated implant size measurements may be usedto identify the angle at which the implant was oriented in relation toan x-ray sensor 312 when the image was produced. The distortion ofimplant dimensions on the radiograph compared with the true proportionsof the implant allow for the calculation of the angle and position ofthe implant in relation to the x-ray source and sensor. Comparison ofthe angle between the implant and x-ray source and adjacent dentalcrowns and the x-ray sources allows for the determination of theangulation of the dental implant in relation to teeth and adjacentstructures, including for example, restorations, bone structure, andperiodontal ligaments. The implant model is identified using, forexample, the Known Implant Measurement Method 370. The identified pixelsrepresenting the implant(s) in the image are identified and totaldistortion of the implant shape is calculated from the change in implantproportions as compared with implant's true dimensional proportions.Total distortion of implant is then used to calculate the angle at whichthe implant was oriented in relation to the x-ray sensor 312 when theradiographic image 120 image was produced.

The calculated angle between implant(s) identified in dental radiograph120 and the x-ray source is compared with the calculated angle betweenadjacent teeth present in the dental radiograph 120 and the x-ray sourcefrom method 350, in order to calculate the angle between the implant(s)identified in dental radiograph and adjacent teeth present in the dentalradiograph.

In step 382, pixels representing a dental implant within theradiographic image 120 as the oral structure 125 image are identified.In step 384, the outer border of the dental implant is identified. Instep 386, the outer border is compared with the Dictionary ofProjections 352 to determine the angle of the implant and the plane ofthe sensor 312 with respect to the x-ray source. In step 388, theposition of the dental implant in relation to other teeth is determinedby comparing the implant source angle to the teeth source angle. Giventhe detailed geometric information available about the dental implantand the angle of the sensor 312, the pixel to millimeter ratio 385 canthen be determined with the highest accuracy.

Referring back to FIG. 5, the collective outputs of the selectedcombination of methods 304, 320, 350, and 370, and 380 comprise thesensor pixel to mm ratios 385, any of which can be selectively employedbased on desired measurement accuracy and their availability from eachmethod, along with the image metadata that is in terms of pixels, toproduce a calibrated measurement 390 that provides desired measurementsof the oral structures 125 in terms of millimeters. Depending on theavailable combination of methods 304, 320, 350, 370, and 380 to providethe sensor pixel to mm ratio 385, step 392 provides a measurementconfidence indication which in turn is used to determine a threshold394.

In some embodiments, confidence metrics are used to determine whichtooth is used for calibration of relative to absolute measurements usingvarious factors (e.g., ranked standard deviation of tooth anatomy foreach given tooth type, presence of previous dental restoration, andpairwise analysis of ratios between available teeth within the imagethat may be used to complete the calibration process and to determineoutliers and discard them from the calibration process). Kalmanfiltering techniques may also be used to incorporate multiple sources ofinformation and make use of all available measurements even if they maybe noisy. At least some of this information can be incorporated with aknown uncertainty to weight their contribution. Confidence ofcalibration 392 is used to adjust threshold 394 for accepting orrejecting the presence of bone loss. Further use of ML techniquesprovided by training mechanism 140 may further enhance the reliabilityand confidence level of measurements based on further use of availableinputs applied in various combinations of available information, such asdata related to previous radiographic images, measurements, and oralstructures for particular patients that take their historicalinformation into account, as well as information on particular sensortypes (such as particular known characteristics that could createnoise).

The library 150 is collectively a data library that may be used to allowthe various methods featured in the system 100 to be performed. Thelibrary 150 may be comprised of data outputted by earlier executions ofthe above-described measurement methodologies, or may be available fromthird-party sources.

Types of information within the library 150 may include, but are notlimited to:

1. X-ray sensor database

-   -   Sensor names    -   Sensor size    -   Dimensions of effective areas (length and width)

Dental Sensor Effective Effective Data Name Manufacturer area lengtharea width Sensor 1 “33”, size 2 Schick _._mm   _._mm _ _._mm  ._mmSensor 2 “Dream DentiMax   _._mm _   _._mm _ Sensor”,  ._mm  ._mm size 02. Implant database;

-   -   Implant names    -   Implant specifications    -   Manufacturer    -   Model name    -   Years produced    -   Dimensions    -   Descriptive attributes        Once the implant is identified using the database, the pixel to        mm ratio of the image can be determined.

3. Population-based Anatomical Averages

-   -   Average dental crown dimensions

Average Crown Mesiodistal Width at CEJ Level Tooth #1 Tooth #2 Etc. Male_._mm _._mm _._mm Female _._mm _._mm _._mm

-   -   Average CEJ-bone level distance

Average CEJ- Alveolar Crest Distance Tooth #1 Tooth #2 Etc. Male _._mm_._mm _._mm Female _._mm _._mm _._mm

-   -   Average dental root dimensions

Average Tooth #3 Tooth #3 Tooth #3 Tooth #4 Root palatal mesiobuccaldistobuccal buccal Length root root Root Root Etc. Male _._mm _._mm_._mm _._mm _._mm Female _._mm _._mm _._mm _._mm _._mm4. Patient-specific data

-   -   Previous 2-D radiographs        -   Folder of 2-D image files (.jpg, .png, etc.)        -   Table of Measurements gathered from processed images    -   Previous 3-D radiographs        -   Folder of 3-D files (DICOM, NRRD, etc.)        -   Table of Measurements gathered from processed images    -   Previous 3-D surface scans        -   Folder of 3-D surface files (.stl)        -   Table of Measurements gathered from processed images

FIG. 13 is an example GUI (graphical user interface) screen on thedisplay 160 for allowing a user to view the dental radiographic image120 in a display area 404 that may include various oral structures 125.A set of graphical measurements may be superimposed on the dentalradiographic image 120 in the display area 404 that correspond to a setof calibrated measurements 390 that are displayed in a data area 400.The set of calibrated measurements 390 may be divided according to thethreshold 394 that is determined in the measurement processor 300.Alternatively, the GUI screen may provide further user controls such asthe ability for the user to set the threshold 394 manually as shown bythe THRESHOLD 3MM control. A user control area 402 includes various GUIscreen functions that may include Hide, Discard, and Save functionsrelated to the displayed measurement results.

FIG. 14 is a block diagram 500 illustrating an example softwarearchitecture 502 that the system 100 may execute on, various portions ofwhich may be used in conjunction with various hardware architecturesherein described (including for the utilization integrity detectionand/or fraud detection implementations described below). FIG. 14 is anon-limiting example of a software architecture and many otherarchitectures may be implemented to facilitate the functionalitydescribed herein. The software architecture 502 may execute on hardwaresuch as client devices, native application provider, web servers, serverclusters, external services, and other servers. A representativehardware layer 504 includes a processing unit 506 and associatedexecutable instructions 508. The executable instructions 508 representexecutable instructions of the software architecture 502, includingimplementation of the methods, modules and so forth described herein.

The hardware layer 504 also includes a memory/storage 510, which alsoincludes the executable instructions 508 and accompanying data. Thehardware layer 504 may also include other hardware modules 512 that mayinclude one or more graphics processing units (GPU), such NVIDIA™GPU's), and may also include special purpose logic circuitry, e.g., anFPGA (field programmable gate array), an ASIC (application-specificintegrated circuit), a DSP processor, an accelerated processing unit(APU), an application processor, customized dedicated circuitry, etc.,to implement, at least in part, the processes and functionality for theimplementations, processes, and methods described herein. Instructions508 held by processing unit 506 may be portions of instructions 508 heldby the memory/storage 510.

The example software architecture 502 may be conceptualized as layers,each providing various functionality. For example, the softwarearchitecture 502 may include layers and components such as an operatingsystem (OS) 514, libraries 516, frameworks 518, applications 520, and apresentation layer 544. Operationally, the applications 520 and/or othercomponents within the layers may invoke API calls 524 to other layersand receive corresponding results 526. The layers illustrated arerepresentative in nature and other software architectures may includeadditional or different layers. For example, some mobile or specialpurpose operating systems may not provide the frameworks/middleware 518.

The OS 514 may manage hardware resources and provide common services.The OS 514 may include, for example, a kernel 528, services 530, anddrivers 532. The kernel 528 may act as an abstraction layer between thehardware layer 504 and other software layers. For example, the kernel528 may be responsible for memory management, processor management (forexample, scheduling), component management, networking, securitysettings, and so on. The services 530 may provide other common servicesfor the other software layers. The drivers 532 may be responsible forcontrolling or interfacing with the underlying hardware layer 504. Forinstance, the drivers 532 may include display drivers, camera drivers,memory/storage drivers, peripheral device drivers (for example, viaUniversal Serial Bus (USB)), network and/or wireless communicationdrivers, audio drivers, and so forth depending on the hardware and/orsoftware configuration.

The libraries 516 may provide a common infrastructure that may be usedby the applications 520 and/or other components and/or layers. Thelibraries 516 typically provide functionality for use by other softwaremodules to perform tasks rather than interacting directly with the OS514. The libraries 516 may include system libraries 534 (for example, Cstandard library) that may provide functions such as memory allocation,string manipulation, file operations. In addition, the libraries 516 mayinclude API libraries 536 such as media libraries (for example,supporting presentation and manipulation of image, sound, and/or videodata formats), graphics libraries (for example, an OpenGL library forrendering 20 and 30 graphics on a display), database libraries (forexample, SQLite or other relational database functions), and weblibraries (for example, Web Kit that may provide web browsingfunctionality). The libraries 516 may also include a wide variety ofother libraries 538 to provide many functions for applications 520 andother software modules.

The frameworks 518 (also sometimes referred to as middleware) provide ahigher-level common infrastructure that may be used by the applications520 and/or other software modules. For example, the frameworks 518 mayprovide various graphic user interface (GUI) functions, high-levelresource management, or high-level location services. The frameworks 518may provide a broad spectrum of other APIs for applications 520 and/orother software modules.

The applications 520 include built-in applications 520 and/orthird-party applications 522. Examples of built-in applications 520 mayinclude, but are not limited to, a contacts application, a browserapplication, a location application, a media application, a messagingapplication, and/or a game application. Third-party applications 542 mayinclude any applications developed by an entity other than the vendor ofthe particular system. The applications 520 may use functions availablevia OS 514, libraries 516, frameworks 518, and presentation layer 544 tocreate user interfaces to interact with users.

Some software architectures use virtual machines, as illustrated by avirtual machine 548. The virtual machine 548 provides an executionenvironment where applications/modules can execute as if they wereexecuting on a hardware machine. The virtual machine 548 may be hostedby a host OS (for example, OS 514) or hypervisor, and may have a virtualmachine monitor 546 which manages operation of the virtual machine 548and interoperation with the host operating system. A softwarearchitecture, which may be different from software architecture 502outside of the virtual machine, executes within the virtual machine 548such as an OS 550, libraries 552, frameworks 554, applications 556,and/or a presentation layer 558.

FIG. 15 is a block diagram illustrating components of an example machine600 configured to read instructions from a machine-readable medium (forexample, a machine-readable storage medium) and perform any of thefeatures described herein that any of the various system (including thesystem 100, the system 700 of FIG. 16, and any of the implementations ofFIGS. 19-22) of the present disclosure can execute instructions on. Theexample machine 600 is in a form of a computer system, within whichinstructions 616 (for example, in the form of software components) forcausing the machine 600 to perform any of the features described hereinmay be executed. As such, the instructions 616 may be used to implementmethods or components described herein. The instructions 616 causeunprogrammed and/or unconfigured machine 600 to operate as a particularmachine configured to carry out the described features. The machine 600may be configured to operate as a standalone device or may be coupled(for example, networked) to other machines. In a networked deployment,the machine 600 may operate in the capacity of a server machine or aclient machine in a server-client network environment, or as a node in apeer-to-peer or distributed network environment. Machine 600 may beembodied as, for example, a server computer, a client computer, apersonal computer (PC), a tablet computer, a laptop computer, a netbook,a set-top box (STB), a gaming and/or entertainment system, a smartphone, a mobile device, a wearable device (for example, a smart watch),and an Internet of Things (IoT) device. Further, although only a singlemachine 600 is illustrated, the term “machine” includes a collection ofmachines (e.g., distributed machines) that individually or jointlyexecute the instructions 616.

The machine 600 may include processors 610, memory 630, and 110components 650, which may be communicatively coupled via, for example, abus 602. The bus 602 may include multiple buses coupling variouselements of machine 600 via various bus technologies and protocols. Inan example, the processors 610 (including, for example, a centralprocessing unit (CPU), a graphics processing unit (GPU), a digitalsignal processor (DSP), an ASIC, or a suitable combination thereof) mayinclude one or more processors 612 a to 612 n that may execute theinstructions 616 and process data. In some examples, one or moreprocessors 610 may execute instructions provided or identified by one ormore other processors 610. The term “processor” includes a multicoreprocessor including cores that may execute instructionscontemporaneously. Although FIG. 15 shows multiple processors, themachine 600 may include a single processor with a single core, a singleprocessor with multiple cores (for example, a multicore processor),multiple processors each with a single core, multiple processors eachwith multiple cores, or any combination thereof. In some examples, themachine 600 may include multiple processors distributed among multiplemachines.

The memory/storage 630 may include a main memory 632, a static memory634, or other memory, and a storage unit 636, both accessible to theprocessors 610 such as via the bus 602. The storage unit 636 and memory632, 634 store instructions 616 embodying any one or more of thefunctions described herein. The memory/storage 630 may also storetemporary, intermediate, and/or long-term data for processors 610. Theinstructions 616 may also reside, completely or partially, within thememory 632 and 634, within the storage unit 636, within at least one ofthe processors 610 (for example, within a command buffer or cachememory), within memory at least one of I/O components 650, or anysuitable combination thereof, during execution thereof. Accordingly, thememory 632 and 634, the storage unit 636, memory in processors 610, andmemory in I/O components 650 are examples of machine-readable media.

As used herein, “machine readable medium” refers to a device able totemporarily or permanently store instructions and data that causemachine 600 to operate in a specific fashion. The term “machine-readablemedium,” as used herein, does not encompass transitory electrical orelectromagnetic signals per se (such as on a carrier wave propagatingthrough a medium); the term “machine-readable medium” may therefore beconsidered tangible and non-transitory. Non-limiting examples of anon-transitory, tangible machine-readable medium may include, but arenot limited to, nonvolatile memory (such as flash memory or read-onlymemory (ROM)), volatile memory (such as a static random-access memory(RAM) or a dynamic RAM), buffer memory, cache memory, optical storagemedia, magnetic storage media and devices, network-accessible or cloudstorage, other types of storage, and/or any suitable combinationthereof. The term “machine-readable medium” applies to a single medium,or combination of multiple media, used to store instructions (forexample, instructions 616) for execution by a machine 600 such that theinstructions, when executed by one or more processors 610 of the machine600, cause the machine 600 to perform and one or more of the featuresdescribed herein. Accordingly, a “machine-readable medium” may refer toa single storage device, as well as “cloud-based” storage systems orstorage networks that include multiple storage apparatus or devices.

The I/O components 650 may include a wide variety of hardware componentsadapted to receive input, provide output, produce output, transmitinformation, exchange information, capture measurements, and so on. Thespecific I/O components 650 included in a particular machine will dependon the type and/or function of the machine. For example, mobile devicessuch as mobile phones may include a touch input device, whereas aheadless server or IoT device may not include such a touch input device.The particular examples of I/O components illustrated in FIG. 6 are inno way limiting, and other types of components may be included inmachine 600. The grouping of I/O components 650 are merely forsimplifying this discussion, and the grouping is in no way limiting. Invarious examples, the I/O components 650 may include user outputcomponents 652 and user input components 654. User output components 652may include, for example, display components for displaying information(for example, a liquid crystal display (LCD) or a projector), acousticcomponents (for example, speakers), haptic components (for example, avibratory motor or force-feedback device and/or other signal generators.User input components 654 may include, for example, alphanumeric inputcomponents (for example, a keyboard or a touch screen), pointingcomponents (for example, a mouse device, a touchpad, or another pointinginstrument), and/or tactile input components (for example, a physicalbutton or a touch screen that provides location and/or force of touchesor touch gestures) configured for receiving various user inputs, such asuser commands and/or selections.

In some examples, the I/O components 650 may include biometriccomponents 656 and/or position components 662, among a wide array ofother environmental sensor components. The biometric components 656 mayinclude, for example, components to detect body expressions (forexample, facial expressions, vocal expressions, hand or body gestures,or eye tracking), measure bio-signals (for example, heart rate or brainwaves), and identify a person (for example, via voice-, retina-, and/orfacial-based identification). The position components 662 may include,for example, location sensors (for example, a Global Position System(GPS) receiver), altitude sensors (for example, an air pressure sensorfrom which altitude may be derived), and/or orientation sensors (forexample, magnetometers).

The UC components 650 may include communication components 664,implementing a wide variety of technologies operable to couple themachine 600 to network(s) 670 and/or device(s) 680 via respectivecommunicative couplings 672 and 682. The communication components 664may include one or more network interface components or other suitabledevices to interface with the network(s) 670. The communicationcomponents 664 may include, for example, components adapted to providewired communication, wireless communication, cellular communication,Near Field Communication (NEC), Bluetooth communication, Wi-Fi, and/orcommunication via other modalities. The device(s) 680 may include othermachines or various peripheral devices (for example, coupled via USB).

In some examples, the communication components 664 may detectidentifiers or include components adapted to detect identifiers. Forexample, the communication components 664 may include Radio FrequencyIdentification (RFID) tag readers, NEC detectors, optical sensors (forexample, one- or multi-dimensional bar codes, or other optical codes),and/or acoustic detectors (for example, microphones to identify taggedaudio signals). In some examples, location information may be determinedbased on information from the communication components 662, such as, butnot limited to, geo location via Internet Protocol (IP) address,location via Wi-Fi, cellular, NFC, Bluetooth, or other wireless stationidentification and/or signal triangulation.

Clinical Data Analysis

As noted, dental-care providers often require pre-authorization forcertain proposed treatment plans. The requests are generally accompaniedby supporting materials, including radiographic image data, to justifythe proposed treatment plans. To provide quick decisions/assessments forthe requests or claims, while avoiding any assessment bias that mayresult from reviewing requests by a large number of different reviewers(as would be required to handle the large volume of expedited requests),proposed herein is an approach that analyzes clinical data (e.g.,information about the procedure treatment plan, radiographic image data,etc.) to determine, in part, whether the proposed treatment is warrantedgiven the source radiographic image data provided with the request (and,optionally, given historical data, such as a patient's previoustreatments and corresponding radiographic images). As will be discussedin greater detail below, an ML model (which may be implemented using asystem architecture similar to that discussed in relation to the imageprocessing module 200 depicted in FIG. 1) is trained to identify andmark various dental features (which may be potentially damaged ordiseased, but which may be healthy) for which a claim or apre-authorization request is submitted (or for which a treatment plan isto be automatically generated), and also identify and mark healthy (atleast partly) dental structures in the image data. The marking(segmentation) of such features is used to derive dimensioned properties(area, length) metrics (e.g., ratios) that can be processed by the sameor another ML engine, or by a rule-based engine, that determines, basedon the proposed treatment plan submitted by the provider, and thederived metrics, the reasonableness or appropriateness of the proposedtreatment. A more comprehensive clinical data analysis that depends onother information and factors (e.g., one that performs one or moreanalysis processes similar to those described below with respect to theFraud and Clinical Data (Utilization) Integrity Detection framework) mayalso be executed to supplement the expedited area-based utilizationanalysis. The inclusion of supplemental clinical data analysisprocessing may depend on a balancing of the expediency desired to reachquick decisions and availability of data needed to carry out thesupplemental processing. In some examples, the implementations discussedherein may also be used by individual dental-care providers to determinepossible treatment plans to use in relation to detected conditions.

Accordingly, the implementations described herein include a method forclinical (diagnostic or utilization) data analysis that includesobtaining treatment dental data (e.g., from a repository of radiographicimage data, directly from an X-ray imaging sensor, or through any othermeans) for an individual, with the dental data (sometimes referred to astreatment clinical data) including input radiographic image data for atleast one dental object (e.g., a tooth, or a number of teeth). Themethod additionally includes identifying, by a learning machine, atleast one first dental feature (e.g., some abnormality, such as acavity, a decaying portion of a tooth, a restoration, bone loss, etc.)in the input radiographic image data for the at least one dental object,indicative of the existence of a dental clinical condition for the atleast one dental object, and at least one other feature (e.g., clinicalcrown portion of the tooth, also referred to as coronal area) in thedental object comprising at least partly a healthy dental structure. Themethod further includes computing dimensions properties (such as areasor lengths, e.g., through pixel counting) for the at least one firstdental feature and the at least one other feature comprising at leastpartly the healthy dental structure, and deriving based on thedimensioned properties one or more ratio metrics (e.g., a ratio of areacovered by abnormal/diseased dental feature, to the area covered by afeature that includes a healthy portion) indicative of severity of thedental clinical condition for the at least one abnormal dental featureof the at least one dental object. In some embodiments, based on acomparison of the derived metric(s), a dental action (e.g., a decisionby the insurance company to deny or accept the claim or request fortreatment plan pre-approval) can be determined.

As described herein, because a ratio metric is a unit-less scalar valuethat does not require data to be measured in a specific length unit, insome embodiments, clinical data analysis to derive metric ratios basedon area computation for various detected features does not necessarilyrequire pre-calibration or pre-standardization of the source image data.Nevertheless, when a more comprehensive analysis is to be performed tosupplement the use of ratio metrics, or when the derived metrics are notnecessarily based on ratios, calibration of input data may need to beused. Furthermore, because embodiments that are based on ML modelimplementations (e.g., to detect healthy and diseased dental features)may have been trained using image data that has been calibrated in somefashion, to avoid the skewing of derived output due to an input runtimeimage not being calibrated in a similar way to the calibration appliedto training data, the source radiographic input data may first becalibrated. Calibration procedures may include the calibration processesdiscussed herein in relation to FIGS. 6-12, as well as otherstandardization processes such as image alignment processes.Accordingly, in some embodiments, the clinical data analysis approachesdescribed herein may further include applying a calibration procedure tothe source radiographic image data to perform one or more of, forexample, the known sensor measurement process, the oral structuremeasurement process, the 3D surface data measurement process, the knownimplant measurement process, and/or the implant angle measurementprocess.

Thus, with reference to FIG. 16, a block diagram of an example system700 to perform clinical data analysis (and which may be configured toperform an expedited analysis that provides decisions for claims orpre-approval treatment requests within seconds or minutes of thesubmission of such request) is shown. Some elements of the frameworkdepicted in FIG. 16 may be similar to the system architectureillustrated in FIGS. 1A and 1B (e.g., the elements to realize theML-model implementation and training aspects of the architecture of FIG.16). The system 700 includes a dental feature detector 710 (which may beimplemented using a machine learning engine realized, for example, usingneural networks or other types of learning machines, or implemented asan image processing/filtering engine) to identify objects in an image,label or mark such identified objects with, for example, geometricshapes indicative of the outlines/contour of the objects. The generatedmasks may be closed geometrical shapes (such as rectangles, or polygonalshapes) enclosing an area, or covering a length, that is used todetermine metrics to assess the reasonableness of treatments proposalssubmitted by dental-care providers.

As also illustrated in FIG. 16, in embodiments in which amachine-learning implementation is used to perform dental featuredetection (and generate masks or labels for the detected features) atraining data repository 712 provides the training data that comprisesradiographic images that have been labelled and/or marked with masksrepresentative of detected dental features. The training data isgenerally constructed with input from human observers, and defines theground truth for the particular ML model that is being implemented bythe ML-based dental feature detector. In some embodiments, the trainingdata stored in the repository 712 is provided to a learning enginecontroller/adapter 714 (which may be similar in implementation and/orfunctionality to the training mechanism 140 of the system 100 shown inFIGS. 1A and 1B) configured to determine and/or adapt the parameters(e.g., neural network weights) of the learning engine that would produceoutput representative of detected objects and/or the generated masks(e.g., geometric shapes or polygonal representations) determined for thesource image data. To train the dental feature detector 710, trainingdata comprising representations of objects/structures (diseased andhealthy dental objects) imaged for different individuals is provided tothe controller/adapter 714. The training data also includes label datarepresentative of the classifications of such objects/structures (e.g.,identifying the nature of the detected feature as being normal orabnormal, what part of the tooth anatomy the feature is, the toothnumbering for the teeth shown in the image, etc.) The representationdata, label data, and other information included with the training datathus define samples of the ground truth that is used to train the dentalfeature detector 710 of the system 700 (offline and/or during runtime).This training data is used to define the parameter values (weights,represented as the vector θ) assigned to links of, for example, a neuralnetwork implementation of the learning engine implementation for thedetector 710. The weight values may be determined, for example,according to a procedure minimizing a loss metric between predictionsmade by the neural network and labeled instances of the data (e.g.,using a stochastic gradient descent procedure to minimize the lossmetric). The computed parameter values can then be stored at a memorystorage device (not shown) coupled to the dental feature detector 710and/or to the controller/adapter 714. After a learning-engine basedimplementation of the detector 710 has become operational (following thetraining stage) and can process actual runtime data, subsequent run-timetraining may be intermittently performed (at regular or irregularperiods) to dynamically adapt the detector 710 to new, more recenttraining data samples in order to maintain or even improve theperformance of the detector 710.

As illustrated in FIG. 16, a communication interface 702 receives asinput source radiographic image data, e.g., x-ray images communicateddirectly from an image apparatus (such as an X-ray sensor) orcommunicated from some intermediary central node or image repository.The data received at the communication interface 702 is generallydirected to the detector 710, optionally via a calibration and imagealigner unit 730, e.g., in situations where the source images need tofirst be calibrated/normalized to determine the appropriatepixel-to-standard-unit length scale, and to align the source image in amanner consistent with the way the training data and images from othersources have been calibrated and/or aligned. Intermittently, thecommunication interface 702 may also receive and forward to the trainingdata repository 712 additional training data to perform updated training(at periodic or irregular intervals) on the learning enginecontroller/adapter 714.

Alternatively or additionally, in some embodiments, the detection ofobjects/features may be implemented using a filter-based approach, inwhich the input image to be analyzed is provided to a filter-baseddetector and mask generator, e.g., to detect shapes and objects in theimage through, for example, feature detection filtering (to detectedges, corners, blobs, etc.), morphological filtering, etc., andgenerate respective outlines or geometric shapes representative of thedental structures, objects, or features detected in the scene.

The output of the dental feature detector 710 may include the generatedmasks or labels superimposed on the source input image to thus providean output image that includes both the actual image data and the masks(e.g., geometric shapes or other representations generated for thedetected dental features in the radiographic image). Examples of MLmodels that may be used to generate masks or marking of dental featuresare discussed throughout the present disclosure. Alternatively, theoutput image can include only the resultant mask (be it geometric shapesor outlines), arranged in a manner that maintains the relativepositioning and orientation of the structures/objects in the originalimage relative to each other. Complex irregular polygonal shapes (tooverlay or replace the actual raw data) can be derived for featuresappearing in the image based on optimization processes, for example, anoptimization process that fits the best closed polygon to a detectedobject, subject to certain constraints (e.g., minimal line length foreach segment of the polygon, minimal area enclosed by the polygon, etc.)

FIG. 17 includes an example dashboard 750 displaying a radiographicimage on which resultant output markings have been superimposed on anoutput image 760 (produced from a source input image) that includes, inaddition to the raw input radiographic image data, rectangular masksrepresentative of an area occupied by detected features. In the exampleof FIG. 17, two features are detected. The first is an abnormal dentalfeature 762, which in this case is the filling (restoration) area andadjacent area that includes a decay tooth matter. The second markedfeature is a dental feature 764 that comprises, at least in part, ahealthy tooth structure, which in the example of FIG. 17 is the clinicalcrown of the tooth (i.e., the visible portion of the tooth above the gumline). Although the image 760 includes rectangular masks that completelyenclose the contours of the detected features, more complex shapes maybe used (e.g., complex polygonal shapes that trace, subject to certainconstraints, the contours of the feature). Furthermore, the resultantoutput may also include labels identifying the detected features (e.g.,labelling the feature 762 as “abnormal” or “restoration+decay,” andlabelling the marked feature 764 as “crown”). It is to be noted thatadditional features could have been detected, if desired (as may bepermitted by the user interface realized through the dashboard, and ifsuch detection versatility is supported by the ML model) to, forexample, separately identify and mark the restoration area and decayarea as separate dental features.

Turning back to FIG. 16, following the detection, marking (orsegmenting), and labeling of the detected features in the inputradiographic image data, the resultant output is provided to an areacalculator and decision module 720 which is configured to computedimensions properties (e.g., areas or lengths) of the at least one firstdental feature (e.g., the feature 762 in the example of FIG. 17) and theat least one other feature comprising at least partly the healthy dentalstructure (e.g., the clinical crown feature 764 in the example of FIG.17). The areas of the marked features may be computed through pixelcounting, or, when using simple masks (such as the rectangular masksused in the example of FIG. 17) by determining the dimensions (lengthand width) of each of the rectangles, and computing areas (if needed)based on those determined dimensions. Other techniques to determine orapproximate areas of marking (masks) generated by the detector 710 mayalso be used.

Having computed the dimensioned properties (e.g., areas covered) by themasks (markings) produced by the dental feature detector 710, the module720 is also configured to derive, based on the dimensioned properties, ametric indicative of severity of the dental clinical condition for theat least one first (e.g., abnormal or damaged) dental feature of the atleast one dental object. For example, a ratio metric may be computed asa ratio of area covered by abnormal (diseased) dental feature (in theexample of FIG. 17, the diseased area is the area enclosed by therectangle 762 marking the restoration+tooth decay portion in the toothanalyzed), to the area covered by a feature that includes a healthyportion (e.g., the clinical crown area enclosed by the rectangle 764.Other metrics may also be computed instead of, or in addition to, thearea ratio metric, including, for example, distances or lengths ofcertain features (e.g., distances between CEJ and bone point on bothsides of a tooth, and CEJ and apex points; these distance metricsindicate if there is bone loss, etc.) The derived ratio metrics cansubsequently be used to make a decision (using a rule-based procedure,or using another ML model implementation that may be independent of theML model used to implement the dental feature detector 710). Thedecision (or some other output generated by the module 720) may berepresentative of a treatment plan to treat the dental issues that mayhave identified through operation of the detector 710 and the module720, or may be a decision responsive to a treatment recommendation (or apre-approval request for a proposed treatment plan) submitted as part ofthe clinical data by the dental-care provider treating the patient whoseradiographic image data is being analyzed. For example, consider anexample where, in addition to the radiographic image shown in FIG. 17,the clinical data also includes a proposed treatment plan to place aprosthetic crown over the tooth being analyzed, due to the size of theexisting restoration and the expanding tooth decay adjacent to therestoration. Assume that in this example, the patient's dental insuranceapproves claims for synthetic crowns when the damage to the naturaltooth matter is more than 40% of the clinical crown area. Based on thiscriterion or rule, the dental provider's treatment plan to treat thetooth for a prosthetic crown fitting would be denied because the ratioof the area of the damaged tooth matter (enclosed by the rectangle 762)to the clinical crown area is less than 40% (as illustrated in FIG. 17,the ratio of area 762 to the area 764 is less than 25%). The area ratiocriterion is but one example of rules that can be implemented via thesystem 700 of FIG. 16, and many other criteria, which may result inelaborate decision schemes, may also be realized. It is to be noted thatthe module 720 may be part of the detector 710, and does not necessarilyneed to be implemented as a dedicated module that is separate from thedetector 710.

Optionally, as also illustrated in FIG. 16, a calibration and imagealigner unit 730 may be located upstream of the dental feature detector710 to calibrate source image data for consistency with the trainingdata used to train the ML model(s) of the system 700, and for greateruniformity of the image data provided from the numerous differentsources (e.g., different dental-care providers). Calibration of thesource image data (e.g., to derive a measurement scale for the images,expressed as a pixel-to-standard-unit ratio, e.g., pixel-to-mm ratio)may also be needed if severity metrics (to analyze and assess clinicaldata) that are not based on ratios (and in which the scale of the imageswould not be immaterial as a result of using ratio metrics) are used.Calibration of the source image data may be performed based on themeasurement (calibration) processes discussed in relation to FIGS. 6-12.Additional details of the calibration processing performed by thecalibration image aligner 730 are provided below in relation to FIG. 18.

If orientation and positioning information for the incoming inputradiographic image data is available or can be derived (e.g., based onone or more of the measurement processes of FIGS. 6-12) to obtain ordetermine viewing angle and distance of an X-ray imaging apparatus fromthe imaged dental objects, the image data can be re-aligned to a commonframe of reference. Such an alignment can further improve the accuracyand consistency of the images being analyzed, thus improving therobustness of the feature identification and decision making performedby the system 700. Based on the available or derived orientation andpositioning information for the sensor apparatus imaging thecraniofacial structures, geometrical corrections for a source image canbe performed to rotate, or otherwise transform the image's pixels,according to various formulations. For example, the relative perspectiveof an X-ray imaging apparatus can be used to derive (using anoptimization process to compute transformation parameters) atransformation matrix that is applied to the source image data to yielda resultant image corresponding to a particular viewing perspective.

With reference next to FIG. 18, a flowchart of an example procedure 800for clinical data (also referred to as utilization or diagnostic data)analysis, generally performed at a computing system such as the system700 of FIG. 16, is shown. The procedure 800 includes obtaining 810dental data for an individual, with the dental data including inputradiographic image data for at least one dental object. Dental data maybe provided directly from a dental-care provider (via one or moreintermediary nodes in a communications network such as the Internet) orfrom some central repository (possibly associated with an insurancecompany to whom treatment plans or claims are first submitted).

The procedure 800 further includes identifying 820, by one or moremachine learning models (such as the ML-based implementation of thedental feature detector 710 of FIG. 16), at least one first (abnormal ordamaged, but potentially healthy) dental feature in the inputradiographic image data for the at least one dental object (that featuremay be indicative of the existence of a dental clinical condition forthe at least one dental object), and at least one other feature in theat least one dental object comprising at least partly a healthy dentalstructure (e.g., a portion of a tooth that is not diseased, restored, ordamaged). For instance, the at least one first dental feature mayinclude one or more of a decaying tooth portion for a tooth, a fillingregion for the tooth, a restoration, and/or bone loss, and the at leastone other feature may include one or more of, for example, a clinicalcrown structure for the tooth, and/or root length.

In some examples, identifying, by the learning machine, the at least onefirst dental feature and the at least one other feature may includegenerating masks, by the learning machine, representative of the atleast one abnormal dental feature and the at least one other featurecomprising at least partly the healthy tooth structure.

With continued reference to FIG. 18, the procedure 800 additionallyincludes computing 830 at least one dimensioned property representativeof physical dimensions of the at least one first dental feature and theat least one other feature comprising at least partly the healthy dentalstructure. The procedure 800 additionally includes deriving 840, basedon the at least one dimensioned property at least one dimensionedproperty ratio indicative of an extent of a dental clinical conditionassociated with the identified at least one dental feature of the atleast one dental object. In some examples, computing the at least onedimensioned property may include computing one or more of areas of theidentified at least one first dental feature and the at least one otherfeature, and/or lengths of the identified at least one first dentalfeature and the at least one other feature. In such examples, derivingthe at least one dimensioned property ratio may include one or more ofderiving an area ratio of an area for the at least one first dentalfeature and area for the at least one other feature, and/or deriving alength ratio of a length of the at least one first dental feature and alength of the at least one other feature.

As further illustrated in FIG. 18, the procedure 800 also includesdetermining 850 a treatment plan based on a comparison of the derived atleast one dimensioned property ratio to a respective at least onepre-determined threshold value. Determining the treatment plan mayinclude one or more of, for example, automatically determining bymachine learning model a proposed treatment plan to treat an abnormaldental feature (this may be done at a dental-care provider's clinic, orat some centralized remote server), and/or determining based on thederived dimensioned property ratio whether to approve adental-care-provider treatment plan submitted by a dental-care provider.

As noted, it may be desirable, in some embodiments, tocalibrate/standardize the received source radiographic image data. Thecalibration processing can include determining the scale (e.g., in somestandard-unit length, such as millimeter) that pixels in the receivedsource image represent. There are several calibration processes that areproposed herein that can be performed without requiring use ofcalibration objects to be included in captured image data. Thoseproposed processes rely instead on archival or other availableinformation about the objects appearing in the captured image, and/orinformation about the sensor devices that are used for capturing theimage data. Thus, in such embodiments, obtaining the clinical data mayinclude receiving source radiographic image data represented accordingto pixel-based dimensions, and calibrating the source radiographic imagedata to produce the input radiographic image data represented in termsof estimated standard-unit dimensions, with the source radiographicimage data being free of any non-dental calibration objects. Theestimated standard-unit dimensions may include millimeter (mm) units.

Calibrating the source radiographic image data may include selecting asegmenter and/or an object detector, predicting source masks and sourcepoints (and/or keypoints) of the at least one dental object appearing inthe source radiographic image data using the segmenter and the objectdetector, providing the source radiographic image data and the imagemetadata, comprising the source masks and source points, to acalibration process selector, selecting by the calibration processselector at least one measurement process from a set of measurementprocesses according to the source radiographic image data and the imagemetadata, deriving a sensor pixel-to-standard-unit ratio using theselected at least one measurement process, and generating the inputradiographic image data and resultant calibrated metadata, whichincludes the calibrated masks and points on the dental object, usingcalibrated measurements of the at least one dental object based on thesensor pixel-to-standard-unit ratio and the image metadata. Anycombination of the following proposed measurement (calibration)approaches may be used (depending on the desired accuracy and availabledata).

A first example of a measurement (calibration) process that does notrely on a dedicated non-dental (i.e., an artificial object that is notpart of the naturally occurring dental structure of a person) is theknown sensor measurement process. Under this proposed process, derivingthe sensor pixel-to-standard-unit ratio using the selected at least onemeasurement process may include determining a sensor type for the sourceradiographic image data, determining sensor characteristics based on thedetermined sensor type, determining pixel dimensions for the sourceradiographic image data, and deriving the sensor pixel-to-standard-unitratio based on the determined sensor characteristics and the determinedpixel dimensions for the source radiographic image data.

A second example of a measurement (calibration) process is the oralstructure measurement process. In this proposed approach, deriving thesensor pixel-to-standard-unit ratio using the selected at least onemeasurement process may include identifying, from the sourceradiographic image data, teeth without restorations, determiningdistances in pixels between mesial and distal Cemento Enamel Junction(CEJ) points for the identified teeth, deriving a plurality ofpixel-to-standard-unit ratios using the determined distances in pixelsand based on pre-determined standard average distances between themesial and distal CEJ points for each of the identified teeth, andcomputing an average pixel-to-standard-unit ratio from the derivedplurality of pixel-to-standard-unit ratios.

A third example of a measurement process is the 3D surface datameasurement process. In this proposed approach, deriving the sensorpixel-to-standard-unit ratio using the selected at least one measurementprocess may include determining one or more outer borders for respectiveone or more dental objects appearing in the source radiographic imagedata, and comparing the one or more outer borders to 2D projections in aprojection dictionary, the 2D projections being at incremental distanceand angles generated from 3D dental image data, to identify a matchbetween the one or more outer borders and the 2D projections in theprojection dictionary. The proposed third example measurement processfurther includes estimating a viewing angle at which the sourceradiographic image data was obtained based on the identified matchbetween the one or more outer borders and the 2D projections in theprojection dictionary, and deriving the sensor pixel-to-standard-unitratio based on the estimated angle at which the source radiographicimage data was obtained.

A fourth example of a measurement process is the known implantmeasurement process. In this proposed approach, deriving the sensorpixel-to-standard-unit ratio using the selected at least one measurementprocess may include detecting an implant structure appearing in thesource radiographic image data, and determining implant attributes basedon the source radiographic image data for the detected implantstructure. The proposed fourth example measurement process furtherincludes comparing the determined implant attributes for the detectedimplant structure to stored implant attributes included in implant datarecords, maintained in an implant structure database, for knownmanufactured implants to identify a match between the determined implantattributes and the stored implant attributes included in the storedimplant data records. The fourth approach also includes deriving thesensor pixel-to-standard-unit ratio based on stored geometricalinformation associated with a selected one of the implant data recordsthat most closely matches (e.g., based on some similarity ormathematical distance criterion) the implant attributes determined fromthe source radiographic image data.

A fifth example of a measurement process is the implant anglemeasurement process. In this proposed approach, deriving the sensorpixel-to-standard-unit ratio using the selected at least one measurementprocess may include detecting an implant structure appearing in thesource radiographic image data, determining implant attributes based onthe source radiographic image data for the detected implant structure,and comparing the determined implant attributes for the detected implantstructure to stored implant attributes included in implant data records,maintained in an implant structure database, for known manufacturedimplants to identify a match between the determined implant attributesand the stored implant attributes included in the stored implant datarecords. The fifth proposed approach also includes determining an outerborder for the detected implant structure appearing in the sourceradiographic image data, comparing the outer border to 2D projectionsmaintained in a projection dictionary, the 2D projections being atincremental distance and angles generated from 3D dental image data, toidentify a match between the outer border and the 2D projections in theprojection dictionary, estimating a viewing angle at which the sourceradiographic image data was obtained based on the identified matchbetween the outer border and the 2D projections in the projectiondictionary, and deriving the sensor pixel-to-standard-unit ratio basedon the estimated angle at which the source radiographic image data wasobtained, and based on stored geometrical information associated with aselected one of the implant data records that most closely matches theimplant attributes determined from the source radiographic image data.In some examples of the fifth proposed approach, deriving the sensor thesensor pixel-to-standard-unit ratio may include estimating viewingangles for other dental objects detected in the source radiographicimage data based on a position of the implant structure relative to theother dental structure and based on the viewing angle at which thesource radiographic image data was obtained, and deriving the sensorpixel-to-standard-unit ratio based on the estimated viewing angles forthe other dental structures detected in the source radiographic imagedata, and based on the stored geometrical information associated withthe selected one of the implant data records that most closely matchesthe implant attributes determined from the source radiographic imagedata.

In some embodiments, the procedure 800 may further include determining atreatment plan to treat the identified at least one abnormal dentalfeature based on the derived severity metric. Determining the treatmentplan may include determining based on the derived severity metricwhether to approve a dental-care-provider treatment plan submitted by adental-care provider.

Fraud and Clinical Data Integrity Detection Framework

As noted, in some embodiments, utilization integrity detectionapproaches are implemented using, for example, multisystem machinelearning to identify anomalous or suspicious dental records (e.g., toidentify suspicious data records representative of treatment utilizationplans submitted by dental-service providers). Integrity detection isapplied to data associated with utilization of services such ashealthcare services in general, and dental services in particular. Thehealthcare services can include other services such as specialtyservices, therapies, and so on. As will be discussed in greater detailbelow, the integrity detection seeks to evaluate, among other things,whether reported treatment clinical data accurately reflects thattreatment for an individual was indicated, and that an appropriatetreatment was provided to the individual. The analysis of the treatmentclinical data further seeks to identify utilization anomalies. Theutilization anomalies include manipulated data, duplicated data, andother threats to utilization integrity. The manipulated and theduplicated data include manipulated dental images and duplicated dental(oral) images, respectively. The identification of manipulated orduplicated clinical data is generally not obvious or easy to detect.While an image that can be provided for clinical data may be uploadedmore than once in error, an image for one individual might be presentedas an image for a second individual. Further, data such as image datacan be manipulated. The image manipulations can include adjusting imageexposure, contrast, highlights, or shadows; flipping an image so that animage of a right side of the individual can be presented as a left sideimage; scaling of the image so that the image is “zoomed in” or “zoomedout”, and so on. While a human observer may be able to notice such datamanipulations, analysis techniques based on algorithms or heuristicsalone can have great difficulty in doing so. Instead, machine learningtechniques are applied to the analysis.

In disclosed techniques, machine learning is accomplished using one ormore machine learning systems (such as those described above in relationto FIGS. 1-18), including based on different types of neural networks.The neural networks can include generator neural networks anddiscriminator neural networks within one or more generative adversarialnetworks (GANs). The generator attempts to create data, called syntheticdata, which is able to fool the discriminator into thinking that thedata is real. The discriminator tries to detect all synthetic data andlabel the synthetic data as fake. These adversarial roles of thegenerator and discriminator enable improved generation of syntheticdata. The synthetic data may supplement training data labelled by humanobservers and may be used to enhance training of the machine learningneural networks. The neural networks training can be based on adjustingweights and biases associated with layers, such as hidden layers, withinthe neural network. The results of the neural network training based onthe augmenting with the synthetic data can be used to further train theneural networks, or can be used to train an additional neural networksuch as a production neural network. The training can be based ondeterminations that include true/false, real/fake, and other types ofoutputs (e.g., decision output to indicate approval or denial ofpre-approval requests submitted by dental-care providers). Trainedneural network can be applied to a variety of analysis tasks includinganalysis of dental radiographic image data.

Neural networks, such as a convolutional neural network, a recurrentneural network, a feedforward network, a transformer network, and so on,can be used to perform machine learning, deep learning, etc. A neuralnetwork for machine or deep learning can be trained by using a machinelearning system to process training data. The training data can includeone or more sets of training data. The training data comprises “knowngood” data, where the known good data includes previously analyzed inputdata and expected results from analyzing the input data (i.e., the“ground truth” data). The known good data is processed by the machinelearning system in order to make adjustments such as adjusting weightsand biases associated with the neural network. Additional trainingadjustments to the neural network can be accomplished by applyingadditional known good data and by making further adjustments to theweights. In embodiments, the training data includes treatment clinicaldata associated with a plurality of individuals. Such training dataincludes radiographic data, where the radiographic data includes dental(oral) images such as x-ray image data. The training data can alsoinclude historic radiography associated with specific individuals,text-based content included within current images and historic images(the text-based content may be extracted using OCR, and the extracteddata provided a text-related ML model), and so on.

Once trained, multisystem machine learning framework described herein isapplied to the treatment clinical data for an individual. As will bedescribed in greater detail below, the multisystem machine learning isused to analyze the data to look for a variety of data anomalies. Thedata anomalies can include treatment outliers, where a treatment outliercan include a treatment which is different from the standard treatmentfor a disease, injury, and so on. The data anomalies can also includetreatment that is determined to be overly aggressive or unwarranted fora given medical condition. The data anomalies can further includeindications that an image, such as a radiographic image or x-ray, hasbeen manipulated or duplicated. The data anomalies can include anomaliesassociated with provider treatments and accuracy.

As noted, the use of radiographic data (e.g., dental x-rays) for bothtraining, and subsequently in runtime to detect anomalous data records,can be facilitated through the image data calibrations and measurementestimation for dental features appearing in image data, according to oneor more of the various measurement estimation approaches described abovein relation to FIGS. 1-15, and may also include image alignmentprocessing.

The comprehensive multisystem machine learning framework describedherein is used for data integrity detection, including for clinical dataintegrity. The utilization refers to a range of treatments provided toan individual. While the present disclosure focuses on dental-relatedtreatments, it will be appreciated that some of the implementationsdescribed herein can be adapted to also analyze treatment data forvarious other treatments and therapies, including physical therapy forrecovery from injury, surgery, or health events such as heart attack orstroke, and so on. Other therapies can include occupational and speechtherapies, and “alternative” therapies such as homeopathic medicine,herbal remedies, meditation, and the like. The implementations describedherein can also be adapted to process or analyze other types ofinsurance claims and coverage (e.g., auto insurance claims, real andpersonal property damage claims, disaster claims, etc.) With respect tothe use of the presently described frameworks to analyze dentaltreatment data, the utilization integrity detection analyzes treatmentclinical data for an individual. The analysis seeks to determine whetherthe treatment offered to an individual—or claimed to have been providedto the individual—was consistent with standard practice for suchtreatments, and also seeks to determine if the images themselves areanomalous (e.g., they are near duplicates of a previously stored images,or they may have been manipulated in some way). While many physicians,healthcare practitioners and workers, therapists, pharmacists, andothers scrupulously report the treatments provided to the individual,others do not. Data associated with a treatment, such as a number oftreatments, the aggressiveness with which a condition is treated, imagessuch as radiographic images, etc., can be “juked” or intentionallymanipulated to deceive an auditor who would review the treatmentclinical data. Phantom treatments provided for phantom diseases can bepresented with the intention to defraud. The analysis of the treatmentclinical data by the implementations described herein can thus be usedto generate scores and metrics that identify phantom claims of diseaseand treatment, determine treatment or disease “outliers”, find overlyaggressive treatments, etc.

Machine learning techniques are applied to the utilization integritydetection. The machine learning can be performed on a processor networkconfigured as a neural network. Examples of machine learning systemconfigurations are also discussed above in relation to the calibrationprocesses (e.g., with respect to FIGS. 1A and 1B) and in relation toclinical data analysis (e.g., with respect to FIG. 16). When an ML modelare implemented using neural networks, those neural networks use dataprocessing nodes interconnected by links (arcs) used for data transferand communication between and among the nodes. To accomplish machinelearning, the neural network must first be “trained”. As discussedherein, training of a neural network can be accomplished by providingground truth data (also referred to as “known good” data) to the neuralnetwork, where the ground truth data includes input data and verifiedinferences associated with the data, and supplemental synthetic trainingdata. It is to be noted that the training of machine learning model canalso be performed using self-supervised models that need little trainingdata and create results by leveraging vast unlabeled data. Parametersassociated with the neural network, such as weights and biases, areadjusted so that the neural network that is being trained makes theverified inferences about the data. Successful training of a neuralnetwork typically requires large amounts of training data to improve thesuccess rate of correct inferences. For the example of utilizationintegrity detection, the training data can be based on treatment dataassociated with a large number of individuals. The training data maycontain manipulated images as well as proper or unaltered ones, falseand true reports of treatments, and so on. The training data includestreatment data, radiographic images such as oral x-rays, etc. Oncetrained, the machine learning is applied to actual data. The “learning”functionality of the ML implementations may continue (intermittently) asthe additional data is processed. The neural network is adjusted or“learns” so that the accuracy of the analysis increases, and inferenceconvergence by the neural network is hastened.

Treatment clinical data can be obtained from a system which includesdata associated with the individual, treatment data, and so on. Thetreatment data generally comprises radiographic data. The radiographicdata can be obtained using ionizing and nonionizing radiographictechniques. The obtaining further includes obtaining data from one ormore additional utilization systems concerning the individual. Theadditional utilization systems can include insurance provider systems,healthcare provider systems, databases such as medical or dentaldatabases, etc. As will become apparent below, one or more reviewmetrics for an individual are generated, where the review metrics arebased on radiographic data analysis (e.g., performed by an MLimplementation). The analysis of data can include determination ofwhether the underlying data is itself compromised (i.e., the data isfraudulent). Examples of compromised image data include radiographicdata that has been manipulated, represents duplicate data, and the like,and may thus be associated with a fraudulent claim. Review metricscorrespond to analysis result of the necessity (justifiability) ofcertain clinical treatment procedures. In some instances, a utilizationsystem score is computed, based on data from at least one of the one ormore additional utilization systems (the utilization system score may bedetermined independently of the review metrics). Examples of the systemscores include metrics, where the metrics can include an outlier metric,an aggressiveness metric, etc. A composite utilization system score canbe derived based, for example, on a weighted average of the outliermetric and the aggressiveness metric. A provider score (also referred toas a clinical data score) is calculated, based on the review metric andthe utilization system score. The clinical data score can be used todetermine if the clinical data associated with an individual indicatesoutlier treatment, overly aggressive treatment, etc. An ongoing analysisof the treatment clinical data is performed, based on additionaltreatment clinical data for the individual. The additional treatmentdata can include data from a plurality of individuals, longitudinaltreatment data for the individual, etc. Additional radiographic dataused in the data processing and analysis can be obtained for a pluralityof other individuals while undergoing commensurate treatment to theindividual. The commensurate treatment can include surgeries, implants,fillings, etc. The additional radiographic data can be taken from theindividual while the individual is undergoing different treatment from atreatment indicated by the treatment clinical data. The additionaltreatment clinical data associated with the individual can be taken froma prior or subsequent dental experience. The ongoing analysis isperformed in real-time or near real-time.

Thus, with reference to FIG. 19, a flow diagram 900 to illustrateexample operations of a multisystem machine learning framework toperform clinical data integrity and fraud analysis is shown. The flowdiagram 900 is based on a computer-implemented method for integrity andfraud analysis, where the clinical-related analysis can be performed onone or more treatments provided to an individual. As illustrated, theflow 900 includes accessing clinical data 910 for an individual. Theclinical data can include various types of data associated with one ormore treatments provided to the individual. When applied to dentalrelated data analysis, the clinical data includes dental treatmentplans, various records and information recorded during dentistappointments (including the dental-care providers' notes) for anindividual, relevant supporting materials such as one or moreradiographic images, etc. Additional data that may be relevant to theanalysis of the dental-related data, or that may be processed todetermine utilization integrity metrics for non-dental treatments, mayinclude data pertaining to inpatient or outpatient treatments such assurgeries or nonsurgical procedures, therapies such as physical,occupational, or speech therapy, data collected or obtained for theindividual for general internal medicine or specialist medicinetreatments, and so on. It is to be noted that an individual'sutilization data may be generated at a provider's location, andtransferred directly to a centralized network node(s) on which thepresent proposed framework is implemented. An individual's utilizationdata may alternatively be transferred to a third-party's (such as aninsurance provider) computing system for storage, and later forwardedtherefrom (along with collected data for other individuals) to thecentralized system (e.g., a cloud server) on which the frameworkperforming the flow 900 is implemented. The centralized system may beoperated by a third party performing the data analyses discussed hereinfor multiple clients (e.g., multiple different insurance providers eachservicing multiple patients and/or multiple dental-care providers).

As further shown in the flow 900, the treatment clinical data comprisesradiographic data 912. The radiographic data can be based on datacollected using ionizing or nonionizing radiation. The radiographic datacan include x-ray data, computed tomography (CT) data, magneticresonance imaging (MRI) data, ultrasound (nonionizing) data, etc. Inembodiments, the radiographic data comprises oral images of theindividual. The oral images can be based on various oral images such asbitewing images, periapical images, occlusal images, and the like. Inthe flow 900, the accessing may also obtain data from one or moreadditional utilization systems 914 concerning the individual. Theadditional utilization systems can include systems associated withinsurance providers, healthcare providers, and so on. The additionalsystems can include scheduling systems to show individual patientappointment schedules, previous and current medical providers, etc.

The flow 900 additionally includes generating 920 a review metric forthe individual 920, where the review metric may be based on radiographicdata analysis. The radiographic data analysis can include analyzing theradiographic data to identify features within the data, artifacts withinthe data, and so on. Such features may be indicative of fraudulentbehavior. As noted herein, the generating may include a process tocalibrate source radiographic images (as more particularly described inrelation to FIGS. 1-16) and/or to perform expedited clinical dataanalysis based, for example, on area-based metrics for features detectedin the radiographic images (as more particularly described in relationto FIGS. 16-18). In some embodiments, the generating aradiographic-based metric can be based on identifying anomalies withinthe data. The review metric can be based on a value, a range of values,a percentage, a threshold, and so on. The metric can be text based,where the text could include “pass” or “fail”, “good” or “bad”, “OK” or“Further examination needed”, etc.

In the flow 900, several processes may be performed to generating theradiographic image data review metric (e.g., in order to detectpotential fraudulent claims). One such process includes the phantomdisease review (analysis) 922. The phantom disease review can includeaccessing an insurance database to access provider data, accessing theprovider data to access patient data, and accessing the patient data.Accessing the patient data can include accessing patient images,appointment schedules, treatment history, current provider information,previous provider information, and the like. The phantom disease reviewcan be used to determine whether the treatment clinical data indicatesthat an individual received treatment for a disease that is notpresented by the individual. In the flow 900, the radiographic data canthus be processed to calculate (derive), at box 924, a phantom diseasereview score. The phantom disease review score can be based on a value,a range of values, a percentage, a threshold, etc. The phantom diseasereview score may, in some situation, also be derived based on detectionof duplicate images included within claims or other data submission. Itis to be noted, though, that data duplication (particularly imageduplication) is typically a separate type of data analysis generallyperformed independently of phantom disease or phantom treatment reviews.

In some examples, generating (at 920) the review metric may also bederived through phantom treatment review process 926. For the phantomtreatment review, the radiographic data is used to calculate (at 928) aphantom treatment review score. As will be described in greater detailbelow, the phantom treatment review score can be based on a numericvalue or evaluation, text, etc. The phantom treatment review score canindicate inconsistent treatment clinical data. The inconsistenttreatment utilization can include treatments other than standardtreatments, too many treatments, too few treatments, and the like.

In some embodiments, if the output of the process to generate the reviewmetric indicates that potential fraudulent images have been detected (asa result of identifying near duplicate images or possible imagemanipulation), the flow 900 may terminate at that point withoutproceeding to the downstream clinical data analysis processes. This isbecause the identification of potentially compromised image data makesfurther analysis of the claim unnecessary (because the claim is alreadyfound to likely contain fraudulent content).

The flow 900 further includes generating 930 a utilization system scorebased on data from at least one of the one or more additionalutilization systems. The utilization system score can be based on one ormore metrics, as discussed below. The utilization system score can beused to evaluate the clinical data, such as the treatment clinical data,and to determine the veracity of the data. In embodiments, the data caninclude errors such as mislabeled data, treatments entered with theincorrect treatment code, and so on. The utilization system score can bebased on one or more values, a threshold limit, a percentage, aprobability, an inference, and the like. The utilization system scorecan be used to identify altered images (such as radiographic images),intentionally misleading data, duplicate data, etc. The utilizationsystem score (computed at 930) can include an outlier metric for thetreatment clinical data. The outlier metric can be based on whethertreatment clinical data associated with an individual tracks withtreatment clinical data of a plurality of other individuals who havebeen provided a similar treatment, or not (e.g., is an outlier). In someembodiments, the utilization system score can also include anaggressiveness metric for the treatment clinical data. An aggressivenessmetric can be based on providing an “aggressive” treatment to anindividual when a more conservative or less complicated treatment wouldhave sufficed. The aggressiveness metric can indicate a more extensivetreatment, a greater number of treatments, and the like. Furtherembodiments may also include computing a false representation metric forthe radiographic data of the individual based on the dental (oral)images with manipulations that display false representations. A falserepresentation can include an image which has been manipulated to show atreatment that was not performed, an image which has been lightened ordarkened, a flipped image, and the like. The false image can includeduplicate images.

The flow 900 further includes computing (at 940) metrics. The metricscomputed at 940 can be based on a weighted average (derived at 944, asmore particularly discussed below) of the outlier metric and theaggressiveness metric described in relation to the operations at 930 ofthe flow 900). The weighted average can also be based (take into accountor combine) other metrics such as a false image metric, a duplicateimage metric, and the like.

The flow 900 includes computing (at 950) a clinical data score based onthe review metric (derived according to the phantom disease review andthe phantom treatment review) and the utilization system score (whichcan combine metrics derived from the aggressiveness and outlierreviews). The clinical data score, as with other scores, can include oneor more values, thresholds, percentages, probabilities, and the like.The clinical data score can be used to compare utilization by anindividual to utilization by a plurality of other individuals. The datascore can be used to determine whether an individual is receiving moreor less treatment for a given condition or disease compared to the otherindividuals. The clinical data score can be used to determine whetherthe claims for treatment are fraudulent.

In the flow 900, generating a review metric (as performed, for example,at 920 of the flow 900 of FIG. 19), generating the utilization systemscore (as performed, for example, at 930), and calculating the clinicaldata score (as performed, for example, at 930 and 940 of the flow 900)may be performed through machine learning implementations (illustratedat 952). The machine learning implementations at 952 may be realized ina manner similar to the implementations (configuration and architecture)used for the image processing unit 200 of FIGS. 1A and 1B, and/or thedental feature detector 710 (and possibly the area calculator anddecision module 720) of the system 100 of FIG. 16. The machine learningimplementations at 952 of FIG. 19 can be based on deep learning. Themachine learning can be implemented using a network such as a neuralnetwork, where the neural network can be trained and optimized to learnto make inferences about the review metric, the utilization score, andthe clinical data.

As further depicted in FIG. 19, the flow 900 includes performing (at960) an ongoing analysis of the treatment clinical data, based onadditional obtained treatment clinical data for the individual. Theongoing analysis can be used to identify trends in the treatmentclinical data, anomalies in the data, treatment “hotspots” orconcentrations (where the concentrations can be associated with anoutbreak of a disease), services provided by a provider, and so on. Insome embodiments, the additional treatment clinical data for theindividual comprises longitudinal treatment clinical data for theindividual. The longitudinal treatment clinical data can be based onpast treatments and current treatments. The longitudinal treatment datacan include images associated with the individual. In embodiments, thedental images comprise current and historic radiography of theindividual. In some embodiments, the ongoing analysis can be based onadditional treatment clinical data for the individual that is performedin real-time or near real-time. The real-time or near real-time analysiscan be used to quickly identify anomalies, outliers, manipulated images,and so on. In some embodiments, the ongoing analysis can be performedautonomously. The autonomous analysis can be accomplished using machinelearning (which may also be implemented using machine learning processessimilar to those performed at 952) where the machine learningimplementation provides progressively improved output for the analysisover time.

The flow 900 further includes calculating a provider score 970 for atreatment provider. The provider score associated with a provider can becompared to the provider scores of a plurality of other providers. Theprovider score can be used to determine whether the provider isrecommending treatments at a rate similar to treatment rates of otherproviders for a given condition, disease, and so on. The provider scoremay be derived based on at least some of the metrics and scores computedat upstream modules of the module computing the provider score. However,in some embodiments, the provider score may be determined at an earlierpoint than that shown in the flow 900, and may be based on fewer oradditional metrics and scores derived in accordance with the utilizationintegrity and fraud detection analysis performed through the flow 900.In the example embodiments described herein, the treatment providersupplies dental treatment to the individual (i.e., the provider is adental-care provider). However, as noted, the framework illustrated inthe flow 900 can be adapted to analyze data for other types oftreatments, therapies, surgeries, and so on, and for the respectiveproviders of such other treatments, therapies, surgeries, etc.

In some embodiments of the framework depicted in FIG. 19, some of theprocesses (including ML-based analyses to detect various features, or tocompute various metrics) need to be performed in sequence since theoutput of some processes is required as input for computation of anotherprocess. For example, in some embodiments, detection of outright fraud(e.g., through detection of near-duplicate images and image manipulationinstances, as more particularly discussed below) may be performed aheadof other clinical data (e.g., utilization) analysis (to determineclinical necessity or reasonableness of treatment plans). If fraud isdetected during this upstream stage of the analysis, there is no need toperform the downstream clinical data analysis (i.e., the medical meritsof a claim are irrelevant if the claim is determined to be fraudulent).In another example, computation of the clinical data score is based on acomputed review metric and a computed utilization system score. In asimilar vein, the provider score is generally dependent on earliercomputed metrics (as more particularly discussed with respect to FIG.21, below) such as the outlier metric, aggressiveness metric, phantomdisease score and phantom treatment score. The provider score process isthus located downstream the processes that compute the metrics andscores on which the provider score process depends. Nevertheless,various steps in the flow 900 (or of any of the other frameworks andflows described herein may be changed in order, repeated, or omitted,without departing from the disclosed concepts. Various embodiments ofthe flow 900 can be included in a computer program product embodied in anon-transitory computer readable medium that includes code executable byone or more processors.

Once predictions resulting from the various models (applied to inputradiographic images), various dimensional properties (areas, distances,and ratios) are computed. For example, distances may be derived betweenvarious keypoints. In some embodiments, for each tooth, distancesbetween cej and bone point are computed on both sides, and cej and apexpoints. These precise computations can be used to indicate if there isbone loss, or there is some other dental abnormality or damage.

As discussed herein, in some embodiments, areas for various dentalfeatures are derived. Some examples of dental features for which theirareas may be computed include:

-   -   Total area of the region (a line between CEJ's created and above        that is the region);    -   Areas of decay;    -   Areas of fillings; and/or    -   Areas missing tooth matter (i.e., broken tooth).

The above example areas can then be used to calculate, for instance, aDMF ratio which is the sum of decay, missing, and filling, divided bythe total coronal area. This gives a number in the range of [0, 1],where 0 indicates there is no decay, missing tooth portion, or filling,whereas 1 indicates the entire coronal area is either missing, includesa decaying tooth region, or has a filling. In the event that the modelscannot identify (predict) both CEJ's of a tooth, an UNKNOWN decision isreturned for the tooth.

Using the above measurements, a decision may be made for a claim or arequest for pre-approval that includes one of the following decisionsfor a claim: Approve, Deny, Review, Downcode, or Request for moreinformation. The decision-making process may be based on an intricatedecision tree or algorithm.

With reference next to FIG. 20, a flow diagram for an example procedure1000 for analyzing radiographic data is shown. Radiographic data, suchas x-ray data, can be associated with an individual. The radiographicdata can be analyzed to determine whether treatment is needed, whichtreatment is indicated, and so on (this can also include the processesdiscussed above in relation to FIGS. 16 and 18). The analysis of theradiographic data can also be used to determine whether falserepresentations are being made, whether inappropriate treatments arebeing recommended by a provider, and so on. The analyzing ofradiographic data can be performed by machine learning implementations,and can be used in conjunction with, or as part of, the analysisperformed in the example flow 900 of FIG. 19. Analysis of radiographicdata enables multisystem machine learning for utilization integritydetection.

More particularly, the flow 1000 includes accessing treatment clinicaldata 1010 for an individual. The treatment clinical data can be accessedfrom one or more databases, where the databases can be associated withone or more utilization systems. The databases can include localdatabases, remote databases, cloud-based databases, mesh-baseddatabases, etc. The treatment data can also be communicated directlyfrom the dental-care provider (via a network such as the Internet) to acentralized system without being first stored in any intermediate orthird-party database. The treatment clinical data can include variousdata types such as image data, text, audio data, video data, and thelike. The radiographic data can include x-rays (namely, dental x-rays).Further treatment clinical data, including data from one or moreadditional utilization systems concerning the individual, can beaccessed. The additional utilization systems can include current andprevious providers, current and previous employers, and so on.

The flow 1000 further includes comparing (at 2020) the radiographic data220 with additional radiographic data from a plurality of otherindividuals. The radiographic data can include data generated usingionizing and nonionizing radiation sources. The radiographic data caninclude x-ray images, computed tomography (CT) images, ultrasonicimages, and so on. In embodiments, the additional treatment clinicaldata can be taken from a prior dental experience or a subsequent dentalexperience. The additional radiographic data can be obtained from otherindividuals. In embodiments, the additional radiographic data can betaken from the plurality of other individuals undergoing commensuratetreatments to that performed on the individual. The commensuratetreatment can include restorations (i.e., fillings), fitting ofimplants, surgical procedures, etc. In further embodiments, theadditional radiographic data can be taken from the individual while theindividual is undergoing different treatment from a treatment indicatedby the treatment clinical data. In a usage example, the individual canbe undergoing a filling while the treatment clinical data indicates thata root canal procedure was performed. In embodiments, the radiographicdata from the plurality of other individuals includes oral images fromthe plurality of other individuals. The oral (dental) images can includebitewing images, periapical images, occlusal images, etc. For example,the dental (oral) images from the plurality of other individuals caninclude oral images with manipulations. Discussed throughout, the oralimages with manipulations can include images that have been lightened ordarkened, scaled images, flipped images, and the like. The dental imageswith manipulations can display false representations. The falserepresentations can include treatments that were not performed,conditions (medical or dental conditions) not presented by theindividual, etc.

The flow 1000 further includes calculating 1030 a false representationmetric for the radiographic data of the individual based on the oralimages with manipulations that display false representations. The falserepresentation metric can be based on a value, a range of values, athreshold, a probability, an “x out of 10” or similar evaluation, and soon. The false representation metric can be derived based on a derivedoutlier metric (as described in relation to the flow 900 of FIG. 19), anaggressiveness metric, a phantom disease review score, and the like. Thefalse representation metric can be used to evaluate a likelihood that animage such as an oral image is a false image, a manipulated image, aduplicate image, etc.

The flow 1000 further includes analyzing 1040 the radiographic data ofthe individual to determine consistency with other individuals'radiographic data. Determining consistency with the individual's otherradiographic data can include determining whether implants, fillings,bridges, or other treatments that have been provided to the individualin the past are still evident in the oral images or are absent due totreatments such as extractions. In embodiments, the analyzing caninclude phantom disease review. Phantom disease review can be used todetermine whether a disease or condition is presented in an image but isnot actually presented by the individual. In further embodiments, theanalyzing can include phantom treatment review. The phantom treatmentreview can be used to determine whether a treatment that is shown in animage was actually provided to the individual. In a usage example, animage can be manipulated to show a filling, where a filling was notactually provided to the individual. In other embodiments, the phantomtreatment review score may indicate inconsistent treatment clinicaldata. Inconsistent clinical data can include showing that a treatmentsuch as an implant was previously provided, and that the same treatmentwas provided again.

The flow 1000 further includes determining (at 1042) consistency withtreatment data. Determining the consistency of the treatment data caninclude determining consistency with other individuals' radiographicdata. The consistency of the treatment data can include determining thata treatment such as a cleaning is provided once, twice, or more timesper year based on a patient's clinical data and physician, dentist, orother recommendations. Determining consistency can be based on standardtreatment practice, overly aggressive treatment, and the like.

Various steps in the flow 1000 may be changed in order, repeated,omitted, or the like without departing from the disclosed concepts.

With reference next to FIG. 21, a block diagram depicting a framework1100 of processes for computing a provider score is shown. A providerscore can be determined for a provider who provides services,treatments, and so on to an individual. The provider score can be usedto determine an accuracy for utilization integrity of the provider. Theprovider score can be used to identify provider treatment clinical datareporting statistics or trends. The provider score can also be used tocompare a given provider with a plurality of other providers todetermine whether the treatments or services provided by the providerare consistent with accepted practice. Determination of the providerscore can be used by a multisystem machine learning framework forutilization integrity detection. As also discussed in relation to FIG.19, treatment clinical data for an individual is accessed, where thetreatment clinical data comprises radiographic data and data from one ormore additional utilization systems concerning the individual. A reviewmetric is generated for the individual, where the review metric is basedon radiographic data analysis. A utilization system score is generatedbased on data from at least one of the one or more additionalutilization systems.

More particularly, as shown in FIG. 21, a claim record is received (at1110) by a system (realized, at least in part, using one or more MLengines) implementing the procedure 1100. A claim record can include aninsurance claim for a service or treatment provided by a provider, andmay also correspond to a request for pre-approval of a proposedtreatment plan. The claim can be initiated based on input 1112, wherethe input can include a type of service, a name of an individual whoreceived the service, a date of the service, a cost of the service, andso on. Various analyses, processes, and reviews can be performed on theclaim in the course of determining a provider score.

One example process includes an outlier analysis 1115 (which may besimilar to the outlier analysis briefly discussed above in relation tothe operation 930 for generating a utilization system score). Theoutlier analysis can analyze whether the claim is within an appropriaterange of values, below a threshold, etc. An appropriate range can bebased on standard practices associated with the treatment. The outlieranalysis can be based on data obtained from an insurance database 1116.The insurance database can include treatment data for a plurality ofindividuals who have received a given treatment. Further data within theinsurance database can include provider data, standard or typicaltreatment costs, a provider “watch list” for unscrupulous providers,etc. In embodiments, the outlier analysis can produce an outlier metric1120 for the treatment clinical data. The outlier metric can include arange of values, a threshold, a percentage, a ranking, and the like. Insome embodiments, the outlier metric (score) may be computed as theWeighted Moving Average (WMA) of weights applied to individual dentalratios categorized by upcoding, unbundling (e.g., instead of chargingfor one procedure, charging for multiple procedures), andoverutilization. Computation of the outlier score can include ananalysis of the number of procedures performed, by the particularprovider, for a particular patient, and how much was charged for theparticular patient.

The framework 1100 for determining the provider score may furtherinclude a utilization review 1125. The utilization review can be used todetermine a frequency of treatment received by an individual, afrequency of a given treatment provided by the provider to a pluralityof individuals, and so on. The utilization review process can calculateor generate a utilization system score. In embodiments, the utilizationsystem review can identify an action 1126, where the action can includean action taken by an insurance provider (e.g., accept a claim, deny aclaim, or partially deny a claim).

The utilization review can also determine an aggressiveness metric (at1130). The aggressiveness metric, representative of how aggressively agiven provider provides a particular treatment, is computed based, inpart, on the output of the utilization review 1125 (e.g., based on thedetermined action 1126). In some examples, the aggressiveness metric isdetermined by how many claim lines do not meet insurance guidelines. Theinput for this review is the claim with supporting materials attachment.The system implementing the flow 1100 is configured, as part of theutilization review process 1125, to process received image data (throughan appropriate ML model) and any text-based content such a provider'snarratives (e.g., using natural language processing) to determinewhether the claim 1110 meets insurance guidelines. For example, for aD4342 procedure (see Appendix A, below), one tooth needs to have bonelevel greater than a specific threshold or presence of subgingivalcalculus (Appendix A, provided below, includes example procedure codesand descriptions that may be used in the performance of a clinical dataanalysis as discussed in relation to the process 1125 of FIG. 21, or theprocedure 800 of FIG. 18). The ML model to process radiographic imagedata to compute a utilization score (which may include the processingdescribed in relation to the system 700 and the procedure 800) canidentify relevant dental features in the radiographic image data, anddetermine the bone level of the affected tooth to thus determine whetherthe threshold and criteria are met. This results in an acceptance ordenial of claim. The percentage of denials is then used to determine theaggressiveness metric. An example formulation to compute anaggressiveness metric is the following:

Aggressiveness score=WMA(*based on time) moving average based on extentof denial ratio*procedure_weights*(1−claims_threshold)

In the above formulation, the denial ratio is computed as the ratio ofthe number of denials (for a particular provider in the presentinstance, or over some period of time) to the number of claim linereviews.

As additionally depicted in FIG. 21, the provider score determinationcan include a phantom disease review process 1135. The phantom diseasereview can determine whether a claimed disease is in fact indicated bydata such as radiographic data. The phantom disease review can beperformed on images 1136, where the images can include oral (dental)radiographic images for the particular individual associated with theclaim record 1110. The oral images can also include current and historicradiography of the individual, oral images from the plurality of otherindividuals (retrieved from a database of images), and so on. Thephantom disease review can generate a flag 113)7. The flag 1137 can beused to indicate that a phantom disease has been found or is suspectedbased on the review. For provider score determination, the radiographicdata is used to calculate a phantom disease review score 1140 (this maybe performed in a manner similar to the process described in relation tothe operations 922 of FIG. 19). The phantom disease review score can bebased on whether a particular disease or diseases are indicated by theradiographic data, or not. In embodiments, the phantom disease reviewscore can include duplicated treatment clinical data. The duplicatedtreatment clinical data may have been entered accidentally orpurposefully.

The phantom disease review score thus represents an indication of aprovider's billing for procedures where a disease (dental condition) wasnot present. The input for this review also includes provider andpatient history databases which have past claims and current images (andother current information, such as chart data, and claim data). Oneanalysis performed in the course of the phantom disease review is todetermine whether the image(s) associated with the claims beingprocessed have been modified (manipulated or tampered) by the provider.

The objective of the image manipulation detection review is to find outthe images that are tampered (e.g., using an image editing tool). Todetermine whether an X-ray image was tampered, the image data (andpossibly its metadata, for intraoral image) is analyzed. Examples ofmanipulation instances are the following:

-   1. Inpainting: In this manipulation, a region of an X-ray is    reconstructed or unwanted parts of an X-ray image are overwritten.    For example, increasing the depth of tooth decay using photoshop    etc.-   2. Copy-move: Copying a patch from an X-ray image and pasting to    another region. This can potentially happen in many ways. For    example, a tooth/part of a tooth can be copied and pasted to another    tooth or bone level can be changed in a similar manner.-   3. Splicing: In this case, an image patch is copied from another    image. For example, a tooth can be copied from another image to hide    something in an X-ray, or to show that there is a problem on a tooth    which needs to be treated etc.

Detection of potential instances of image manipulation may be performed,for example, by inputting a copy of the current source radiographicimage data into an ML model configured to identify potential regions inthe image that may have been manipulated. In some embodiments, theML-based implementation may be configured to detect manipulation usingdental treatment history for the particular patient associated with thesubmitted claim. Thus, an ML engine may be trained to detect imagemanipulation (e.g., as part of a phantom treatment review analysis)using data from the current image and an archival image for the patient.The ML engine may be configured to flag potential regions of differencebetween the current and archived radiographic image that are consistentwith certain types of image manipulation. Consider the following three(3) examples:

A tooth is tampered in a way that it has an additional treatment. Forexample, there is root canal treatment visible in a first, earlier,image (T1) which cannot be seen in the current (future) image (T2) thatincludes image data for the tooth being analyzed. In some embodiments,the analysis may be performed by applying a dental feature detector(which may be similar to the detector 710 of FIG. 16) to generatetext-based labels (and/or additional representations) identifying theaffected tooth and the treatment performed on it. The generated labelsmay then be compared to previously produced labels (e.g., represented asbinary variable for each tooth and/or each treatment) for archivedimages. Alternatively, the labels for archived images may be generatedcontemporaneously.

-   1. A tooth has the treatment which can be seen in both T1 and T2.    However, T2 is not consistent with T1 (and should be impossible to    happen). For example, both images have root canal, but in T1, the    root canal is deeper (longer) than T2 which either indicates some of    the filling is removed from the tooth, or there is some forgery.    This type of image manipulation may be detected through ML-based    image comparison of T1 and T2. The image manipulation analysis may    also be performed by taking into account identified landmarks within    the image that indicate relative positioning and orientations of the    various dental features. The detection of landmarks (e.g., by an ML    model to detect various dental features or landmarks) can increase    the confidence in the detection of potential manipulation since the    detection is not strictly based on deriving feature size or length    (e.g., in mm), but instead also provides relative positioning    information (e.g., does the root canal beyond a certain detected    landmark).-   2. The X-ray belongs to another patient. Regardless of whether    dental treatment history matches, the teeth may be different    (different shaped teeth, different angles between teeth (i.e., more    than θ°), etc.) Detection of this type of manipulation may be    performed based on a structural fingerprinting for the oral    structure of the patient (e.g., tooth1 is associated with a    characteristic (fingerprint) record, or characteristic vector that    may include information such as: {angle:    °, height: h mm, width: w mm, CEJ, other descriptive parameters}).

In some embodiments, image tampering detection can be implemented basedon the heuristic that two x-ray images that are submitted before andafter treatment (possibly taken a few weeks apart) are extremelyunlikely to be the same orientation, angulation, etc., with theexception of the treatment (i.e., it is unlikely that two images of thesame dental object will only be different in the purported treatmentperformed on the dental object; rather, a positioning and perspectivemismatch between the two images is expected). For example, and withreference to FIG. 26A, example pre- and post-treatment x-ray imagesmatches 1602 and 1604 are shown, in which no geometrical mismatch(transformation) is apparent. As seen the biting block and the teeth,bone structure, even the random artifacts are exactly the same. Thepossibility of this happening is extremely unlikely, and thus the rootcanal treatment and the fillings on top of the two left teeth are likelyto have been the result of image tampering (i.e., via photoshop or otherphoto editing tools).

Consider another example, as provided in FIG. 26B, showing pre- andpost-treatment x-ray images 1612 and 1614. In investigating thepost-treatment image investigation of the post-images using digitalforensics tools reveals that the post-treatment image is possiblymanipulated. In some embodiments, tampering detection is based on blindclassification of an image to determine if an image is manipulated usinga DNN architecture. Output of such a classifier can identify whichpixels are potentially manipulated.

In some embodiments, a digital image forensics method can be used forimage tampering detection. These methods can use conventional imageprocessing methods such as detecting the inconsistencies through JPEGcompression artefact or through noise level analysis across an image. Itcan also use a deep neural network such as a CNN based architecture todetect anomalies within an image.

In some embodiments, the methodology of image tampering detection may bea classification method which may indicate whether an image is tamperedor not. Such methods may have a confidence score between 0 and 1 whichis an indication of how likely a given image is manipulated. A score of0 indicates that image tampering has almost certainly not occurred,whereas a score of 1 indicates that there is a 100% confidence levelthat the image has been tampered with.

In some embodiments, image tampering detections may be implementedthrough a detection procedure which indicates the regions on an imagewhere tampering has occurred. The detection output can include abounding box, or may include a segmentation mask that shows exactlywhich pixels in the image likely have been manipulated.

In some examples, image tampering detection procedures may alsoidentify, as part of the detection process output, the tools that wereused for editing the image, such as Photoshop, Paint, Gimp etc.Moreover, the image tampering detection procedures can also indicateimage processing methods that an input image may have undergone,including such output indicating Copy-Move Forgery, Impainting, and soon. These tools may use image content or EXIF header (Exchangeable imagefile format) of the image.

The results of the image tampering detection may be used in PhantomDisease or Phantom Treatment score. This calculation may include theconfidence score of the above image tampering detectors. In some cases,this may require an approval step from a professional such as a dentistor computer vision expert who can confirm that there is potential imagetampering.

In some embodiments, the results of image tampering detectors may beused to estimate provider score. This calculation may use the frequencyof images which are predicted as tampered, the confidence score of eachprediction, whether it is confirmed by an expert, and so on.

Having determined potential instances of manipulated images, an exampleformulation for the phantom disease score is: PD score=WMA of #phantomdisease claims, where the number of phantom disease claims may be thesum of detected instances of claims (submitted for a particular one ormore providers) with supporting images that are likely manipulatedimages. The phantom disease score, which may be computed for particularproviders, can capture the frequency at which phantom diseases aredetected.

In some implementations, the phantom disease review process and/or thephantom treatment process may also include duplicate detection or nearduplicate detection analysis. The findings of these analyses may be usedto calculate a provider score. The duplicate and near duplicatedetection may be performed independently of the provider score analysis,e.g., as an initial step to assess potential fraudulent claims. Theduplicate or near-duplicate analysis yields a duplicate score,indicative of a likelihood that a currently submitted image is aduplicate of a previously submitted image (for the present patient, orfor a different patient). The goal of the duplicate detection review maybe to determine if a claim has been submitted before or may be to detectif a phantom disease was added for a patient. Alternatively, it may beto detect if a phantom treatment was performed for a patient. Detectingthis may not be straightforward as images can be compressed,geometrically transformed (such as resizing, cropping, or flipping),printed and scanned, etc. In such cases, although the claims areperceptually very similar, how they are stored in the disk may becompletely different which requires a deep investigation. An exampleprocess for a near duplicate search process is the following:

-   1. Feature extraction: Using a convolutional neural network    (CNN)-based implementations or another deep neural network    architecture to create one or more feature sets for each image. It    can also use computer vision (CV) techniques such as scale-invariant    feature transform (SIFT), speeded up robust features (SURF),    Oriented FAST and rotated BRIEF (ORB) to create a set of feature    descriptors from each image. Linear Discriminant Analysis (LDA) may    be used as a post processing step for these descriptors. This    functionality aims to decrease the dimensionality of the feature    descriptors while increasing the separability of them from each    other. In other words, similar descriptors will have higher    similarity scores whereas different features will have lower scores.-   2. Extracting signatures from images (using the extracted features):    Many feature descriptors will be created for each image. For each of    these feature descriptors, a signature is created. Additionally or    alternatively, the feature descriptors may be aggregated in order to    generate an aggregate signature.-   3. Creating signature database (i.e., indexing): A process that will    create signatures from all X-ray images and then index those    signatures with the corresponding unique image. Each image may have    multiple signatures.-   4. Searching query images in the database: This step searches for    images that are very similar to source image for which the near    duplicate processing is being performed. Two example ways to do this    include:    -   a) Linear search (pairwise similarity): Under this approach, the        signatures of each of an image are compared to the signatures        stored in a database. This approach is computationally costly        because it requires each signature to be compared with all the        signatures in the database.    -   b) Approximate nearest neighbor search (multi-wise similarity):        Suppose that the database contains n images. Under this        approach, rather than comparing the query image with all n        images contained in the database, the most similar r of them can        efficiently be returned. This is done by creating many “hash        buckets” and the images whose signatures fall into the same        bucket are deemed to be similar. Examples of hash buckets        include: i) Locality Sensitive Hashing, in which random hash        functions are created and these hash functions try to put the        similar images into the same bucket, and ii) Learning to Hash,        which is an NN-based solutions to create hash functions that are        discriminative of similar images/objects.-   5. Comparing query images with the images with high similarity    score: The searching step returns r most similar images for    approximate nearest neighbor search. Some of those images may not be    the same image but very similar ones to the queried image (i.e.,    they are false positives). The comparing step goes through those    near duplicate images and filters out those that do not match to    identify a true actual copy. In this step, each feature point of the    query image can be exhaustively compared with each feature point in    the r most similar images. This is a computationally expensive    operation. Therefore, a method such as k-nearest neighbor search    (knn) or RANdom SAmple Consensus (RANSAC) can be used to compare the    query image with similar images.

Turning back to FIG. 21, the provider score determination may also bebased on a phantom treatment score that is derived from a phantomtreatment review process 1145. The phantom treatment review can be basedon the phantom disease review, on treatment (TX) data 1146 associatedwith a patient x, and so on. The phantom treatment review can be used todetermine whether a treatment was provided, whether the treatment wasthe accepted or standard treatment, and so on. The phantom treatmentreview generates a flag 1147 that is used to indicate that a phantomtreatment is suspected or has been found based on the phantom treatmentreview. The provider score determination can include a phantom treatment(PT) score 1150 determined based on the phantom treatment review 1145.The phantom treatment review score can be based on a threshold, a value,a range of values, an “x out of y” score, and so on. In embodiments, thephantom treatment review score can indicate inconsistent treatmentclinical data. The inconsistent treatment clinical data can indicate toomany or too few treatments, an incorrect treatment, an overly aggressivetreatment, and so on. The treatment review score can be based on thetype of treatment provided. To determine phantom treatments, patienttreatment history may be utilized to determine whether the treatment isshown in future claims. Determination of the phantom treatment score maybe based on the formulation of: PT score=WMA of #phantom treatmentclaims.

The provider score determination can be further based on calculating aprovider score 1170 (where the provider is the individual/entitysubmitting the image), where the provider score can be calculated for aparticular treatment provider. The treatment provider can include aphysical or occupational therapist, a physician, a specialist, and soon. In embodiments, the treatment provider can supply dental treatmentto the individual. The provider score can be calculated based onconsideration of the outlier score 1120, the aggressiveness score 1130,the phantom disease review score 1140, and the phantom treatment reviewscore 1150. For example, the provider score may be computed as weightedsum or average of the outlier metric 1120, the aggressiveness metric1130, the phantom disease score 1140, and the phantom review score 1150,with the weights for each of the score being adjustable weights that canvary based on one or more factors (e.g., the specific provider, thespecific insurance company, etc.) The adjustable weight may themselvesbe derived using, for example, a ML model. Other scores based on otherreviews, analyses, calculations, and so on, can also be included in thecomputation of the provider score, and the provider score may be derivedaccording to various formulations. The provider score can be used togenerate an output 1172. The output can include a value, an answer suchas “yes” or “no”, an approval or rejection to disbursement payment, anapproval or rejection of a request for pre-approval, and so on. Theoutput from the provider score determination can be added to or includedwith the treatment clinical data.

In another example, a provider score (also referred to as a providerrisk/concern score, which may be represented as the score 1170, or someother output) may be derived as a number between 0 and 100, where 0 islow risk and 100 is high risk that their claims should be furtherinvestigated. In such embodiments, additional analysis modules toanalyze data pertaining to claims submitted by a provider include amodule (not shown in FIG. 21) of whether a tooth/quadrant/treatment isnot present. When one or more images are submitted for a claim, but theyeither do not contain the necessary information such as the claimedtooth, or a quadrant (i.e., one of Upper Left Upper Right, Lower Left orLower Right), or a post-treatment image that does not contain thetreatment, those claims are flagged. Another analysis module determinesinconsistencies in tooth history. For example, suppose that a patientundergoes multiple treatments and information about some of thesetreatments is available. The inconsistent history analysis moduleinvestigates if future claims contain the old treatments. For example, apatient's upper left wisdom tooth is removed at time 1, but theextracted tooth is nevertheless visible at a later time 2. In this case,either or both of the treatments are flagged.

FIG. 22 provides further details regarding the phantom disease reviewprocess. In order to reduce the risk of making duplicate payments,erroneous payments, and so on, a phantom disease review can beperformed. A phantom disease can include a disease that is not actuallyassociated with an individual. A phantom disease claim can be made in acomplaint for a patient for a disease or other malady that the patientis not actually experiencing. A phantom disease can result from aprovider entering invalid or inaccurate treatment clinical data (in someexamples, an error can be the result of data integrity errors introducedby the payer where a wrong attachment(s) is associated with a claim,e.g., where another patient's radiographs are sent). Phantom diseasereview allows a multisystem machine learning implementation to perform autilization review and integrity detection analysis. Treatment data foran individual is accessed, where the treatment data comprisesradiographic data (and optionally other data from one or more othersystem collecting data in relation to the individual). A review metricfor the individual is generated, where the review metric is based onradiographic data analysis. A utilization system score is generated,based on data from at least one of the one or more additionalutilization systems. A clinical data score is calculated, based on thereview metric and the utilization system score. An ongoing analysis ofthe treatment clinical data is performed, based on additional treatmentclinical data for the individual.

Phantom disease review can be based on data maintained in insurancedatabases, provider information, patient information, etc., as moreparticularly illustrated in FIG. 22. In some embodiments, phantomdisease review can include accessing an insurance database 1210. Theinsurance database can include treatment clinical data provided by oneor more providers. The providers can include provider A 1212, provider B1214, provider C 1216, and so on. While three providers are shown, othernumbers of providers can be included in the insurance database. Aprovider database such as a database for provider x 1220, can beassociated with each provider within the insurance database. A providercan provide services such as dental services, or other types ofservices, such as physical therapy or occupational therapy services,treatments such as medical treatments, and so on. The treatment providercan supply dental treatment to the individual. The dental treatment thatis provided can include routine dental treatments such as cleanings andfillings, surgeries such as extractions or fitting of implants, etc. Theprovider x database 1220 can include treatment clinical data associatedwith one or more patients. The patients can include patient a 1222,patient b 1224, and so on. While two patients are shown, other numbersof patients can be included.

The phantom disease review can also access a patient database. Thepatient database (such as database 1230) can include treatment(clinical) data associated with one or more patients such as patient x.Various types of data associated with a patient can be included in thepatient database. In embodiments, the data within the patient databaseincludes images 1232. The images associated with the patient can includephotographic images, radiographic data such as x-ray images, chart dataand so on. The data can also include medical/dental chart data (based onwhich inconsistencies between the charts and image modalities can bedetected). In embodiments, the radiographic data comprises oral (dental)images of the individual. The oral images can include images collectedduring treatment of the individual. In embodiments, the oral images caninclude current and historical radiography of the individual. Thephantom disease review can be based on analyzing the images collectedduring treatment of the individual. Embodiments include comparing theradiographic data from the individual with additional radiographic datafrom a plurality of other individuals. The additional radiographic datacan include relevant radiographic data such as radiographic data thatincludes oral images from the plurality of other individuals. Thecomparison can include comparing the radiographic data from theindividual against that data collected from other individuals known toexhibit a given disease. In some examples, the additional radiographicdata can be taken from the plurality of other individuals whileundergoing commensurate treatment to the individual. The treatment caninclude fillings, extractions, implants, etc.

The phantom disease review can be based on the additional radiographicdata. The additional radiographic data can be taken from the individualwhile the individual is undergoing different treatment from a treatmentindicated by the treatment clinical data. That is, the additionalradiographic data can indicate that a treatment claimed to have beenperformed by a provider was not actually provided by the provider byeither sending someone else's data or manipulating the data returning tothe patient x record in the database 1230, the database can includeappointment information 1234. The appointment information can include aspecific appointment, further appointments, and past appointments. Thepatient x database can include treatment history data 1236. Thetreatment history data can include the history of some or all treatmentsprovided to the individual over a time period, where the time period caninclude month, a year, the patient's lifetime, etc. The patient xdatabase 1230 can include current provider information 1238. The currentprovider information can include a provider name, contact informationsuch as phone number, email address, webpage, physical address, and thelike. The patient x database records can include previous providerinformation 1240. The previous provider information can include one ormore providers who provided treatments to the individual and informationassociated with each provider.

FIG. 23A shows an example 1300 of a manipulated x-ray image. Treatmentclinical data associated with an individual can be analyzed to determinewhether the treatment clinical data has been manipulated. The treatmentclinical data can include descriptions of the treatment, images such asphotographic or radiographic images, insurance information, and so on.Data such as radiographic data, image data, etc., can be manipulated.The manipulation of the data may be innocent or unintentional, while atother times the manipulation can include malicious or fraudulentmanipulation. Treatment clinical data for an individual is accessed,where the treatment clinical data comprises radiographic data and datafrom one or more additional utilization systems concerning theindividual. A review metric is generated for the individual, where thereview metric is based on radiographic data analysis. In the example ofFIG. 23A, an original image 1310, such as a radiographic image or x-ray,can include an unaltered or unmanipulated image. The original image canshow a treatment performed. Data associated with the treatment performedcan be included with treatment clinical data. The image can show adental implant 1312. In addition to the dental implant, the image canshow further detail 1314, where the further detail can be associatedwith the tooth to which the dental implant was applied. A second image1320 is also shown in FIG. 23A, where the second image can represent analtered or manipulated version of the first image 1310. The manipulatedimage can show an implant 1322, where the implant 1322 can represent theimplant 1312 seen in image 1310. Note however that other aspects of theimage 1320 have been manipulated. An example of the image manipulationis shown in 1324. The manipulation of the image can include boostingcontrast of the image, manipulating image sharpness, lightening shadowswithin the image, and the like. The image can be manipulated to implythat treatment was performed beyond or in addition to the treatment thatwas actually completed.

FIG. 23B includes examples 1302 of duplicate x-ray image submission.Treatment clinical data can include radiographic data including imagessuch as x-ray images. The radiographic or image data can include imagesof the individual before treatment was performed, after treatment wasperformed, and so on. Duplicate images can be included in the treatmentclinical data by being provided in error, provided for a differentindividual, submitted maliciously or fraudulently, and on. Image 1330shows an original image associated with an individual. The images caninclude an unaltered image such as an x-ray image. Image 1340 shows adarkened version of image 1330. The image 1340 can be darkened bychanging an exposure value associated with the original image. Otheradjustments that can be made can include adjusting contrast, adjustinghighlights, lightening shadows, adjusting sharpness, and other imageadjustments. While black and white images are shown, color images canalso be included in the treatment clinical data. For color images,further adjustments can include adjusting saturation, temperature, tint,etc. The image 1350 shows the image 1330 rotated about a vertical axis,“flipped” left for right, or mirrored. In a usage example, a dentalx-ray flipped about a vertical axis could imply that the treatmentutilization could be associated with an opposite side of the mouth ofthe individual. Such a flipped image could be used to imply that dentaltreatment was applied to both sides of the individual's mouth ratherthan just one side. Image 1360 shows a resized version of image 1330.The resizing of the image can include enlarging the original image,shrinking the original image, and so on. In the example image 1360, theimage has been enlarged. Otherwise, the image 1360 shows substantiallysimilar treatment as that shown in image 1330. Comparison of the alteredimages to the original, whether by darkening, flipping, resizing, and soon, can be accomplished using one or more image processing techniques.In some embodiments, the comparison of the altered images can bedetected using artificial intelligence techniques.

With reference next to FIG. 24, a flowchart of an example procedure 1400for utilization integrity detection is shown. The procedure 1400 may beperformed on a system implementation that is based on some of theframeworks described herein, for example, the frameworks depicted inflows 900 and 1100 of FIGS. 19 and 21, that are configured to performdental data analysis to compute various integrity/veracity metrics withrespect to submitted claims (or requests for treatment pre-approval),including to compute a provider score that identifies or flags anyissues specific to the dental-care provider submitting the claims(including issues identified over time that show problematic trends inthe provider's submitted clinical data for the provider's patients). Theprocedure 1400 includes obtaining 1410 by a computing system dental data(also referred to as clinical data) for an individual, with the dentaldata comprising input radiographic image data of at least one dentalobject (e.g., a tooth), and treatment data representative of one or moretreatment procedures associated with the at least one dental object. Thedental data may be provided directly from a dental-care provider (viaone or more intermediary nodes in a communications network such as theInternet) or from some central repository (possibly associated with aninsurance company to whom treatment plans or claims are firstsubmitted).

The procedure 1400 further includes analyzing 1420, by one or moremachine learning models implemented by the computing system, the inputradiographic image data to identify one or more dental featuresassociated with the at least one dental object. Some of the dataanalysis performed at 1420 may be performed by one or more machinelearning engines (which may be implemented similarly to the ML enginesdiscussed in relation to FIG. 1-15 to perform calibration operations, orthe ML engine implementations depicted in FIG. 16, which uses an MLmodel to identify features in a radiographic image, and determine basedon areas enclosed by those features whether to approve or deny claims orrequests for pre-approval). The features identified by the ML engineinclude various dental features indicative of dental clinical conditions(e.g., decaying portions), dental structures (different parts of thedental anatomy), dental corrective structures (e.g., implants,restorations, root canals, and so on), and labels/descriptors toclassify or describe certain features and their attributes (e.g.,locations and orientations of dental structure). The latter descriptorscan be used to compare an incoming input image to archived images storedat a dental database (e.g., managed by the administrator running theintegrity analysis frameworks described herein). Each type of thevarious features identified (or extracted) can be determined accordingto separate independent ML model implementations trained to identifythat specific type of feature. Alternatively, one or more ML model canbe trained and optimized to identify, and generate appropriate output(masks, marking, labels) for multiple feature types.

In some embodiments, the ML models are implemented to identifydifference between one image and another. Under this ML differentialapproach, the ML engine is configured to receive input corresponding totwo images (e.g., the input radiographic image, and some previouslystored image against which the input radiographic image is beingcompared). It is to be noted that when a differential ML model is used,some pre-processing may be required on at least one of the images. Suchpre-processing may include image calibration (e.g., in accordance withthe implementations discussed in relation to FIGS. 1-15), imagealignment, segmenting, or processing the images to generate compact datarepresentative of the content and/or classification of the data.

As noted, the analysis of the input image data may include detection ofsuspicious anomalies that may indicate image manipulation, or recyclingof a previously submitted image (to support a previous claim) in supportof the current claim. Thus, in such embodiments, analyzing the inputradiographic image data may include detecting anomalous features in theinput radiographic image data, including determining one or more of, forexample, whether a portion of the input radiographic image datasubstantially matches a portion of a previously stored radiographicimage, and/or whether a portion of the input radiographic image data wasmodified. A determination that an input radiographic image containsthese types of anomalies (and may therefore be part of a fraudulentclaim) may result in termination of the remainder of the data analysisprocessing (since a finding of possible fraud would be dispositive ofthe outcome of the claim).

Having performed the learning-machine-based analysis, the procedure 1400further includes deriving 1430, by the computing system, based on thetreatment data and the identified one or more dental features associatedwith the at least one dental object, one or more integrity scores (alsoreferred to as veracity scores) for the input radiographic image dataand the treatment data, with the one or more integrity scores beingrepresentative of potential integrity problems associated with the inputradiographic image data and the treatment data. Deriving the one or moreintegrity scores includes deriving a provider score representative ofpotential integrity problems associated with a dental-care providersubmitting the treatment data.

As also discussed with respect to FIG. 21, the computation of a providerscore may be based on computing various metrics that are eachrepresentative of different potential issues associated with the dental(utilization) data submitted (according to the identified featuresextracted through operation of the one or more learning machinesemployed in the analysis). The provider score can then be computed as acomposite score that combines, under some formulation, the separatelycomputed metrics and output of the dental data analysis. For example,and as noted above, the provider score can be computed as a weighted sum(or weighted sum of averages) of the various metrics computed. In oneexample of a metric computation (for use in computing a compositeprovider score), deriving the provider score may include computing anoutlier score representative of a level of deviation between a treatmentplan, specified in the treatment data for the at least one dental objectto remedy a dental condition identified in the treatment data for theindividual, and treatment plans to treat similar dental conditionsassociated with archived treatment data and archived radiographic imagedata for a plurality of other individuals. In another example, derivingthe provider score may include determining, by the computing system, oneor more possible treatment plans to remedy a dental condition identifiedin the treatment data for the individual, and computing anaggressiveness score representative of a complexity level differencebetween a specified treatment plan submitted by the dental-care providerto remedy the dental condition and the determined one or more possibletreatment plans.

Deriving the provider score may also include, in some embodiments,computing a phantom disease score representative of a level ofconsistency between a treatment plan specified in the treatment data forthe at least one dental object to remedy a dental condition identifiedin the treatment data for the individual, and identified features of theinput radiographic image data detected by the computing system. Thephantom disease score is computed to assess the existence of potentialtampering of the supporting radiographic image data submitted with thetreatment data. In this situation the treatment plan may be consistentwith dental conditions or issues apparent in the radiographic image, butit is suspected that the provider may have submitted compromised imagedata that was altered in some way from the actual radiographic imagedata obtained for individual patient in order to support an unwarrantedtreatment plan. Examples of image data alteration include using at leasta portion of a previously taken x-ray image (for the individual patient,or for some other patient), or manipulating at least a portion of thesubmitted image in some way. Thus, in some embodiments, computing thephantom disease score may include performing image manipulationdetection on the input radiographic image data to determine whether aportion of the input radiographic image data was modified, ordetermining, based on future image data, that the treatment was neverperformed. In some examples, computing the phantom disease score mayinclude performing a near-duplicate image detection on the inputradiographic image data to determine whether a portion of the inputradiographic image data, relating to an identified dental condition forthe at least one dental object, substantially (or fully) matches aportion of a previously stored radiographic image.

Another metric used to assess a provider score is the phantom treatmentscore. Deriving the provider score may include computing, based on thetreatment data associated with the input radiographic image data for theindividual, and based on archived treatment data for one or moreindividuals treated by the dental-care provider, a phantom treatmentscore representative of extent to which the dental-care provider submitstreatment plans inconsistent with associated dental conditionsidentified form the treatment data for the individual and from thearchived treatment data.

As noted, in some situations, the utilization integrity analysis mayalso derive an area-based analysis of dental features detected in aradiographic image (in a manner similar to that discussed in relation toFIGS. 16-18) to determine if a proposed treatment plan is warranted inview of the radiographic image data provided. Thus, in such situations,deriving (in the procedure 1400) the one or more integrity scores mayinclude identifying, by at least one of the one or more machine learningmodels, at least one first dental feature in the input radiographicimage data for the at least one dental object (possibly indicative ofthe existence of a dental clinical condition for the at least one dentalobject), and at least one other feature in the dental object comprisingat least partly a healthy dental structure, computing at least onedimensioned property representative of physical dimensions of the atleast one first dental feature and the at least one other featurecomprising at least partly the healthy dental structure (e.g., computingareas covered, or lengths of identified dental features, or distancebetween two or more identified features such as locations of CEJ lines),and deriving based on the at least one dimensioned property at least onedimensioned property ratio indicative of an extent of a dental clinicalcondition associated with the identified at least one dental feature ofthe at least one dental object. In such embodiments, the analysis mayfurther include determining a treatment plan based on a comparison ofthe derived at least one dimensioned property ratio to a respective atleast one pre-determined threshold value.

The procedure 1400 may also include a radiographic image datacalibration procedure in order to make the source data compatible withthe data that was used train the various ML models implemented for theclinical data integrity analysis, and to more accurately compare theinput image to archived images (at least where the metrics being derivedrequire such a comparison). The calibration processing can includedetermining the scale (e.g., in some standard-unit length, such asmillimeter) that pixels in the received source image represent. As notedabove, the proposed calibration processes are generally performedwithout requiring use of calibration objects to be included in capturedimage data. Those proposed processes instead rely on archival or otheravailable information about the objects appearing in the captured image,or about the sensor devices that are used for capturing the image data.Thus, in such embodiments, obtaining the dental data may includereceiving source radiographic image data represented according topixel-based dimensions, and calibrating the source radiographic imagedata to produce the input radiographic image data represented in termsof estimated standard-unit dimensions, with the source radiographicimage data being free of any non-dental calibration object. The variouscalibration processes used in relation to the procedure 1400 may besimilar to the calibration processes discussed with respect to theprocedure 800 of FIG. 18, and in relation to the various proceduresdiscussed in relation to FIGS. 6-12.

FIG. 25 is a diagram of an example system 1500 to perform utilizationanalysis (the system 1500 may be used, at least in part, to perform theprocedures and processes depicted in FIG. 19-24, or any of theprocedures and processes shown in relation to FIGS. 2-13 and 18). Asdiscussed herein, utilization analysis uses multisystem machine learningfor utilization integrity detection. Alternative implementations ofsystems to perform the procedures and processes described in relation toFIGS. 19-24 include the systems of FIGS. 1A-B, and 14-16. The systemsdescribed in relation to those figures may be modified, as needed, as tobe configured to perform the processes and procedures described inrelation to FIGS. 19-24.

The system 1500 can include one or more processors 1510 attached to amemory 1512 which stores instructions. The system 1500 can include adisplay 1514 coupled to the one or more processors 1510 for displayingdata, intermediate steps, instructions, x-ray images, treatment clinicaldata, and so on. In embodiments, one or more processors 1510 areattached to the memory 1512 where the one or more processors, whenexecuting the instructions which are stored, are configured to: accesstreatment clinical data for an individual, with the treatment clinicaldata including radiographic data and data from one or more additionalutilization systems concerning the individual; generate a review metricfor the individual, with the review metric being based on radiographicdata analysis; generate a utilization system score, based on data fromat least one of the one or more additional utilization systems;calculate a clinical data score, based on the review metric and theutilization system score; and perform an ongoing analysis of thetreatment clinical data, based on additional treatment clinical data forthe individual. In some embodiments, the system is configured to performthe processes 800 and 1400 shown in FIGS. 18 and 21, respectively. Theutilization integrity detection can be used to detect fraudulent data.Fraudulent data can include altered data, duplicate data, and so on. Theutilization integrity detection can be accomplished using the processors1510, computers, servers, remote servers, cloud-based servers, and thelike.

The system 1500 can include a collection of instructions andradiographic data 1520. The instructions and data 1520 may be storedusing techniques such as electronic storage coupled to the one or moreprocessors, a database, one or more code libraries, precompiled codesegments, source code, apps, or other suitable formats. The instructionscan include instructions for generating a review metric based onradiographic data analysis. The radiographic data can include x-raydata. The instructions can also include instructions for generating autilization system score (e.g., veracity/integrity scores) based on datafrom at least one of the one or more additional utilization systems. Theadditional utilization systems can include systems for performingoutlier analysis, reviewing phantom disease or phantom treatment, andthe like. The instructions can include instructions for calculating autilization score based on the review metric and the utilization systemscore. The reviewing can be based on a value, a threshold, a percentage,a range of values, etc. The instructions can include instructions forperforming ongoing analysis of the treatment clinical data based onadditional treatment clinical data for the individual. The data caninclude x-ray data, image data, treatment data, patient medical historydata, physician and healthcare team notes, dentist notes, and so on. Thedata can include data from an insurance database such as provider data.

The system 1500 can include an accessing component 1530. The accessingcomponent 1530 can include functions and instructions for accessingtreatment clinical data for an individual. The clinical data that isaccessed can be available in a local database, a remote, cloud-baseddatabase, a mesh-based database; can be uploaded by a user, can be sentdirectly from a provider's office (including directly from an imagingapparatus that can establish a communication link to a communicationsnetwork), and so on. The clinical data can be encrypted to meet securityand handling requirements such as Health Insurance Portability andAccountability Act (HIPAA) requirements. The treatment information datacan include data contained within one or more insurance databases, oneor more provider databases, patient data, and the like. The treatmentclinical data can include radiographic data and data from one or moreadditional utilization systems concerning the individual. The additionalclinical data for the individual can include medical history data,insurance data, past treatment data, recommended treatment data, and soon.

The system 1500 can include a generating component 1540. In embodiments,the generating component 1540 can include functions and instructions forgenerating a review metric for the individual, with the review metricbeing based on radiographic data analysis. The review metric can be usedto verify veracity of radiographic data, a likelihood that theradiographic data is unaltered or accurate, and so on. In embodiments,the review metric can be based on a value, a percentage, a range ofvalues, a probability, and the like. In embodiments, the generatingcomponent 1540 of the system 1500 can further include functions andinstructions for generating one or more utilization systemscores/metrics, based on data from at least one of the one or moreadditional utilization systems. The one or more such scores can eachinclude a value, a percentage, a range of values, etc. The utilizationscore can include an “out of 10” or “out of 100” score such as 93 out of100. The clinical data score can be used for a variety of applicationsincluding fraud detection, data duplication such as insurance claimsdata, and the like. One example metric that can be generated is anoutlier metric for the treatment clinical data. The outlier metric canbe used to determine whether a treatment for which payment is sought isstatistically different from or substantially similar to treatments forother individuals. As noted, in some embodiments, the system 1500 cancompute an aggressiveness metric for the treatment clinical data. Anaggressiveness metric can be based on whether an “aggressive” approachwas taken for a treatment where a simpler and less expensive treatmentwould have been indicated or would have sufficed. In furtherembodiments, the utilization system can produce a metric that is basedon a weighted average of an outlier metric and an aggressiveness metric.Since an individual can have preexisting health conditions or otherhealth issues that potentially present complications to the treatment,an outlier metric value and an aggressiveness metric do not necessarilyindicate that the utilization was unwarranted. In embodiments, theweighted average of the outlier metric and the aggressiveness metric canbe based on consideration of an individual's other health factors.

The system 1500 can include a calculating component 1550. Thecalculating component 1550 can include functions and instructions forcalculating a clinical data score, based on the review metric and theutilization system score. The utilization system scores can be based ona value, a range of values, a percentage, a threshold, and so on. Theutilization system score can be based on assessments such as “highutilization”, “low utilization”, and “average utilization”. Thecalculating component can be used to calculate further scores. Inembodiments, the radiographic data (discussed previously) allowscomputing a phantom disease review score. A phantom disease can includea disease reported by an individual or a provider, where the disease isnot actually present in the individual. In embodiments, the phantomdisease review score can indicate duplicated treatment clinical data. Aprovider can present treatment clinical data which was also reported atan earlier date, for a different patient, etc. In other embodiments, theradiographic data can be used for computation of a phantom treatmentreview score. The phantom treatment review score can be based on theduplicated treatment, on unwarranted treatment, on incomplete treatment,on overtreatment, etc. In embodiments, the phantom treatment reviewscore can indicate inconsistent treatment clinical data. Inconsistenttreatment data can include treatment data for a different disease,inconsistent processes or application between treatments, and the like.

In embodiments, the treatment provider supplies dental treatments to theindividual. The treatment provider can provide other treatments such asan annual physical, specialized treatments such as dental treatments,ophthalmic, orthopedic, cardiac, oncologic treatment, and the like.Further embodiments can include calculating a provider score for atreatment provider. The provider score can be based on utilizationsystem data (including radiographic image data and treatment data, whichmay be text-based narratives and descriptions of the treatments thatwere performed, or are proposed to be performed) associated with thetreatment provider, and possibly collected over a period of time. Theprovider score can be used to evaluate the treatment provider such asissuing evaluations based on a letter grade. Further embodiments includecalculating a false representation metric for the radiographic data ofthe individual based on the oral images with manipulations that displayfalse representations. False representations can be based on manipulateddata such as manipulated radiographic data. Manipulated radiographicdata can include radiographic images that are duplicated, enhanced,darkened or lightened, enlarged or reduced, reversed (e.g., flipped leftto right), etc.

The system 1500 can further include a performing component 1560. Theperforming component 1560 can include functions and instructions forperforming an ongoing analysis of the treatment clinical data, based onadditional treatment clinical data for the individual. The additionaldata can comprise data collected from a variety of sources. Inembodiments, the additional treatment clinical data for the individualcan include longitudinal treatment clinical data for the individual. Theongoing analysis of the treatment clinical data can be used to determinetrends in treatment such as increases or decreases in treatment,treatment “hotspots” where the number of treatments is elevated above anominal or typical value, and so on. The performing ongoing analysis canbe used to determine increasing or decreasing numbers of fraudulentutilization claims. In embodiments, the ongoing analysis is performedautonomously. The performing autonomous analysis can be based on usingapplications, apps, or codes. The ongoing analysis can be based onartificial intelligence techniques. The ongoing analysis can beperformed online or offline with the generating, calculating, andperforming techniques.

The system 1500 can include a computer program product embodied in anon-transitory computer readable medium for utilization analysis, thecomputer program product comprising code which causes one or moreprocessors to perform operations of the procedures and processesdescribed herein in relation to FIGS. 1-24.

Each of the above methods and processes described herein may be executedon one or more processors on one or more computer systems. Each of theabove methods may be implemented on a semiconductor chip and programmedusing special purpose logic, programmable logic, and so on. Embodimentsmay include various forms of distributed computing, client/servercomputing, and cloud-based computing. Further, it will be understoodthat the depicted steps or boxes contained in this disclosure's flowcharts and flow diagrams are solely illustrative and explanatory. Thesteps may be modified, omitted, repeated, or reordered without departingfrom the scope of this disclosure. Further, each step may contain one ormore sub-steps.

Computer program instructions may include computer executable code. Avariety of languages for expressing computer program instructions mayinclude without limitation C, C++, Java, JavaScript™, ActionScript™,assembly language, Lisp, Perl, Tcl, Python, Ruby, hardware descriptionlanguages, database programming languages, functional programminglanguages, imperative programming languages, and so on. In embodiments,computer program instructions may be stored, compiled, or interpreted torun on a computer, a programmable data processing apparatus, aheterogeneous combination of processors or processor architectures, andso on. Without limitation, embodiments of the present invention may takethe form of web-based computer software, which includes client/serversoftware, software-as-a-service, peer-to-peer software, or the like.

In embodiments, a computer may enable execution of computer programinstructions including multiple programs or threads. The multipleprograms or threads may be processed approximately simultaneously toenhance utilization of the processor and to facilitate substantiallysimultaneous functions. By way of implementation, any and all methods,program codes, program instructions, and the like described herein maybe implemented in one or more threads which may in turn spawn otherthreads, which may themselves have priorities associated with them. Insome embodiments, a computer may process these threads based on priorityor other order.

As noted, the machine learning engines used by various systems andframeworks described herein (as discussed in relation to FIGS. 1-24) maybe implemented as neural networks. Such neural networks may be realizedusing different types of neural network architectures, configuration,and/or implementation approaches. Examples neural networks that may beused include convolutional neural network (CNN), feed-forward neuralnetworks, recurrent neural networks (RNN), transformer network, etc.Feed-forward networks include one or more layers of nodes (“neurons” or“learning elements”) with connections to one or more portions of theinput data. In a feedforward network, the connectivity of the inputs andlayers of nodes is such that input data and intermediate data propagatein a forward direction towards the network's output. There are typicallyno feedback loops or cycles in the configuration/structure of thefeed-forward network. Convolutional layers allow a network toefficiently learn features by applying the same learnedtransformation(s) to subsections of the data. A transformer network is amachine learning configuration (used, for example, in natural languageprocessing and computer vision applications) that includes an attentionmechanism to weight network connections according to their significance.Other examples of learning engine approaches/architectures that may beused include generating an auto-encoder and using a dense layer of thenetwork to correlate with probability for a future event through asupport vector machine, constructing a regression or classificationneural network model that indicates a specific output from data (basedon training reflective of correlation between similar records and theoutput that is to be identified), etc.

Implementations described herein, including implementations using neuralnetworks, can be realized on any computing platform, including computingplatforms that include one or more microprocessors, microcontrollers,and/or digital signal processors that provide processing functionality,as well as other computation and control functionality. The computingplatform can include one or more CPU's, one or more graphics processingunits (GPU's, such as NVIDIA GPU's), and may also include specialpurpose logic circuitry, e.g., an FPGA (field programmable gate array),an ASIC (application-specific integrated circuit), a DSP processor, anaccelerated processing unit (APU), an application processor, customizeddedicated circuit, etc., to implement, at least in part, the processesand functionality for the neural networks, processes, and methodsdescribed herein. The computing platforms typically also include memoryfor storing data and software instructions for executing programmedfunctionality within the device. Generally speaking, a computeraccessible storage medium may include any non-transitory storage mediaaccessible by a computer during use to provide instructions and/or datato the computer. For example, a computer accessible storage medium mayinclude storage media such as magnetic or optical disks andsemiconductor (solid-state) memories, DRAM, SRAM, etc. The variouslearning processes implemented through use of the neural networks may beconfigured or programmed using TensorFlow (a software library used formachine learning applications such as neural networks). Otherprogramming platforms that can be employed include keras (an open-sourceneural network library) building blocks, NumPy (an open-sourceprogramming library useful for realizing modules to process arrays)building blocks, etc.

While the foregoing has described what are considered to be the bestmode and/or other examples, it is understood that various modificationsmay be made therein and that the subject matter disclosed herein may beimplemented in various forms and examples, and that the teachings may beapplied in numerous applications, only some of which have been describedherein. It is intended by the following claims to claim any and allapplications, modifications and variations that fall within the scope ofthe present teachings.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly or conventionally understood. As usedherein, the articles “a” and “an” refer to one or to more than one(i.e., to at least one) of the grammatical object of the article. By wayof example, “an element” means one element or more than one element.“About” and/or “approximately” as used herein when referring to ameasurable value such as an amount, a temporal duration, and the like,encompasses variations of ±20% or ±10%, ±5%, or +0.1% from the specifiedvalue, as such variations are appropriate in the context of the systems,devices, circuits, methods, and other implementations described herein.“Substantially” as used herein when referring to a measurable value suchas an amount, a temporal duration, a physical attribute (such asfrequency), and the like, also encompasses variations of ±20% or ±10%,±5%, or +0.1% from the specified value, as such variations areappropriate in the context of the systems, devices, circuits, methods,and other implementations described herein.

As used herein, including in the claims, “or” as used in a list of itemsprefaced by “at least one of” or “one or more of” indicates adisjunctive list such that, for example, a list of “at least one of A,B, or C” means A or B or C or AB or AC or BC or ABC (i.e., A and B andC), or combinations with more than one feature (e.g., AA, AAB, ABBC,etc.). Also, as used herein, unless otherwise stated, a statement that afunction or operation is “based on” an item or condition means that thefunction or operation is based on the stated item or condition and maybe based on one or more items and/or conditions in addition to thestated item or condition.

Although particular embodiments have been disclosed herein in detail,this has been done by way of example for purposes of illustration only,and is not intended to be limit the scope of the invention, which isdefined by the scope of the appended claims. Any of the features of thedisclosed embodiments described herein can be combined with each other,rearranged, etc., within the scope of the invention to produce moreembodiments. Some other aspects, advantages, and modifications areconsidered to be within the scope of the claims provided below. Theclaims presented are representative of at least some of the embodimentsand features disclosed herein. Other unclaimed embodiments and featuresare also contemplated.

APPENDIX A—EXAMPLE INSURANCE CODES AND DESCRIPTION FOR VARIOUS DENTALPROCEDURES

The following examples of code-specific relationships, if identified inratios above routine clinical use, may raise concern for fraudulent,wasteful, or abusive overutilization. They should be measured againstthe defined policies and administrative systems in place, and the ratiothreshold determined by the benchmark historical and outlier data heldwithin the company.

Diagnostic Services (D0100-0999)

-   -   D0150/0180—Comprehensive Oral Evaluation/Comprehensive        periodontal evaluation        -   Concern: Submitted in lieu of regular periodic exam (D0120),            limited oral exam—problem focused (0140) or            re-evaluation—limited (0170), or assessment of a patient            (0191)        -   Reason: Upcoding services    -   D0210—Complete series of radiographic images        -   Concern: Filing for D0210 when fewer site-specific images            are obtained, such as intraoral—periapical (0220/0230) and            bitewing—single or multiple (0270/0272/0273)        -   Reason: Upcoding services    -   D0274/0272—Bitewing (4 or 2) radiographic images        -   Concern: Filing with two additional periapical images            (D0220/0230) to capture anterior teeth with no medical            necessity        -   Reason: Performing services without medical necessity    -   D0220/0230—Periapical (1 or more) radiographic images        -   Concern: Filing for reimbursement with crown insertion            (D2740/2750) are usually considered working films as part of            the primary procedure of cementing the crown        -   Reason: Performing services without medical necessity

Preventive Services (D1000-1999)

-   -   D1351—Sealant (per tooth)        -   Concern: Filing each sealant as single surface posterior            resin (D2391)        -   Reason: Filing for services not rendered

Restorative Services (D2000-2999)

-   -   D2332—Resin-based composite—three surfaces, anterior        -   Concern: Filing for treatment of Class III restoration (MLF,            DLF), for teeth #6-11 and 22-27        -   Reason: Upcoding services    -   D2335—Resin-based composite—four or more surfaces involving the        incisal angle, anterior        -   Concern: May be filed for wear and abrasion without further            documentation for teeth #6-11 and 22-27        -   Reason: Performing services without medical necessity    -   D2393—Resin-based composite—three surfaces, posterior        -   Concern: Filing multiple teeth in the posterior,            particularly premolars, are suspicious of wear, abrasions            and abfractions (most dental plans do not benefit for            restorations place for wear, abrasions or abfractions)        -   Reason: Performing services without medical necessity    -   D2720/2721—Crown—resin with high noble metal/Crown—resin with        predominantly base metal        -   Concern: Submitting as D2750 (porcelain fused to high noble            metal) to increase reimbursement        -   Reason: Upcoding services    -   D2950—Core Build-up, including any pins where required        -   Concern: Filed with crown codes (D2750, 2740) should not be            routine if replacing minor existing restorations prior to            crown preparation; likely required in less than 50% of crown            restorations (not approaching 1:1)        -   Reason: Upcoding services

Periodontal Services (D4000-4999)

-   -   D4249—Clinical crown lengthening—hard tissue        -   Concern: Submitted in lieu of gingivectomy or gingivoplasty            to allow access for restorative procedure (D4212) when            performed on the same day as crown delivery        -   Reason: Upcoding services    -   D4341/4342—Periodontal scaling and root planning—four or more        teeth/one to three teeth        -   Concern: Submitted in lieu of prophylaxis—adult (D1110)        -   Reason: Upcoding services    -   D4355—Full mouth debridement—to enable a comprehensive oral        evaluation and diagnosis        -   Concern: May be submitted in lieu of a prophylaxis where            there is gross calculus as part of periodic maintenance            rather than to enable evaluation and diagnosis        -   Reason: Upcoding services; Performing services without            medical necessity    -   D4381—Localized delivery of antimicrobial agents via a        controlled release vehicle        -   Concern: If placed routinely without proper diagnostic and            clinical justification.        -   Reason: Performing services without medical necessity

What is claimed is:
 1. A computer-implemented method for dental dataanalysis, the method comprising: obtaining by a computing system dentaldata for an individual, the dental data comprising input radiographicimage data of at least one dental object, and treatment datarepresentative of one or more treatment procedures associated with theat least one dental object; analyzing, by one or more machine learningmodels implemented by the computing system, the input radiographic imagedata to identify one or more dental features associated with the atleast one dental object; and deriving, by the computing system, based onthe treatment data and the identified one or more dental featuresassociated with the at least one dental object, one or more integrityscores for the input radiographic image data and the treatment data, theone or more integrity scores representative of potential integrityproblems associated with the input radiographic image data and thetreatment data, wherein deriving the one or more integrity scorescomprises deriving a provider score representative of a potentialintegrity problem associated with a dental-care provider submitting thetreatment data.
 2. The method of claim 1, wherein deriving the providerscore comprises: computing an outlier score representative of a level ofdeviation between a treatment plan, specified in the treatment data forthe at least one dental object to remedy a dental condition identifiedin the treatment data for the individual, and treatment plans to treatsimilar dental conditions associated with archived treatment data andarchived radiographic image data for a plurality of other individuals.3. The method of claim 1, wherein deriving the provider score comprises:determining, by the computing system, one or more possible treatmentplans to remedy a dental condition identified in the treatment data forthe individual; and computing an aggressiveness score representative ofa complexity level difference between a specified treatment plansubmitted by the dental-care provider to remedy the dental condition andthe determined one or more possible treatment plans.
 4. The method ofclaim 1, wherein deriving the provider score comprises: computing aphantom disease score representative of a level of consistency between atreatment plan specified in the treatment data for the at least onedental object to remedy a dental condition identified in the treatmentdata for the individual, and identified features of the inputradiographic image data detected by the computing system.
 5. The methodof claim 4, wherein computing the phantom disease score comprises one ormore of: performing image manipulation detection on the inputradiographic image data to determine whether a portion of the inputradiographic image data was modified, or determining, based on futureimage data, that the treatment was never performed.
 6. The method ofclaim 4, wherein computing the phantom disease score comprises:performing a duplicate or near-duplicate image detection on the inputradiographic image data to determine whether a portion of the inputradiographic image data, relating to identified dental condition for theat least one dental object, substantially matches a portion of apreviously stored radiographic image.
 7. The method of claim 1, whereinderiving the provider score comprises: computing, based on the treatmentdata associated with the input radiographic image data for theindividual, and based on archived treatment data for one or moreindividuals treated by the dental-care provider, a phantom treatmentscore representative of extent to which the dental-care provider submitstreatment plans inconsistent with associated dental conditionsidentified form the treatment data for the individual and from thearchived treatment data.
 8. The method of claim 1, wherein deriving theone or more integrity scores comprises: identifying, by at least one ofthe one or more machine learning models, at least one first dentalfeature in the input radiographic image data for the at least one dentalobject, and at least one other feature in the dental object comprisingat least partly a healthy dental structure; computing at least onedimensioned property representative of physical dimensions of the atleast one first dental feature and the at least one other featurecomprising at least partly the healthy dental structure; and derivingbased on the at least one dimensioned property at least one dimensionedproperty ratio indicative of an extent of a dental clinical conditionassociated with the identified at least one dental feature of the atleast one dental object.
 9. The method of claim 1, wherein analyzing theinput radiographic image data comprises: detecting anomalous features inthe input radiographic image data, including determining one or more of:whether a portion of the input radiographic image data substantiallymatches a portion of a previously stored radiographic image, or whethera portion of the input radiographic image data was modified.
 10. Themethod of claim 1, wherein obtaining the dental data comprises:receiving source radiographic image data represented according topixel-based dimensions; and calibrating the source radiographic imagedata to produce the input radiographic image data represented in termsof estimated standard-unit dimensions, wherein the source radiographicimage data is free of any non-dental calibration objects.
 11. The methodof claim 10, wherein calibrating the source radiographic image datacomprises: selecting a segmenter and an object detector; predictingsource masks and source points of the at least one dental objectappearing in the source radiographic image data using the segmenter andthe object detector; providing the source radiographic image data andimage metadata, comprising the source masks and source points, to acalibration process selector; selecting by the calibration processselector at least one measurement process from a set of measurementprocesses according to the source radiographic image data and the imagemetadata; deriving a sensor pixel-to-standard-unit ratio using theselected at least one measurement process; and generating the inputradiographic image data and resultant calibrated metadata, comprisingcalibrated masks and points on the dental object, using calibratedmeasurements of the at least one dental object based on the sensorpixel-to-standard-unit ratio and the image metadata.
 12. The method ofclaim 11, wherein deriving the sensor pixel-to-standard-unit ratio usingthe selected at least one measurement process comprises: determining asensor type for the source radiographic image data; determining sensorcharacteristics based on the determined sensor type; determining pixeldimensions for the source radiographic image data; and deriving thesensor pixel-to-standard-unit ratio based on the determined sensorcharacteristics and the determined pixel dimensions for the sourceradiographic image data.
 13. The method of claim 11, wherein derivingthe sensor pixel-to-standard-unit ratio using the selected at least onemeasurement process comprises: identifying from the source radiographicimage data teeth without restorations; determining distances in pixelsbetween mesial and distal Cemento Enamel Junction (CEJ) points for theidentified teeth; deriving a plurality of pixel-to-standard-unit ratiosusing the determined distances in pixels and based on pre-determinedstandard average distances between the mesial and distal CEJ points foreach of the identified teeth; and computing an averagepixel-to-standard-unit ratio from the derived plurality ofpixel-to-standard-unit ratios.
 14. The method of claim 11, whereinderiving a sensor pixel-to-standard-unit ratio using the selected atleast one measurement process comprises: determining one or more outerborders for respective one or more dental objects appearing in thesource radiographic image data; comparing the one or more outer bordersto 2D projections in a projection dictionary, the 2D projections beingat incremental distance and angles generated from 3D dental image data,to identify a match between the one or more outer borders and the 2Dprojections in the projection dictionary; estimating a viewing angle atwhich the source radiographic image data was obtained based on theidentified match between the one or more outer borders and the 2Dprojections in the projection dictionary; and deriving the sensorpixel-to-standard-unit ratio based on the estimated angle at which thesource radiographic image data was obtained.
 15. The method of claim 11,wherein deriving a sensor pixel-to-standard-unit ratio using theselected at least one measurement process comprises: detecting animplant structure appearing in the source radiographic image data;determining implant attributes based on the source radiographic imagedata for the detected implant structure; comparing the determinedimplant attributes for the detected implant structure to stored implantattributes included in implant data records, maintained in an implantstructure database, for known manufactured implants to identify a matchbetween the determined implant attributes and the stored implantattributes included in the stored implant data records; and deriving thesensor pixel-to-standard-unit ratio based on stored geometricalinformation associated with a selected one of the implant data recordsthat most closely matches the implant attributes determined from thesource radiographic image data.
 16. The method of claim 11, whereinderiving a sensor pixel-to-standard-unit ratio using the selected atleast one measurement process comprises: detecting an implant structureappearing in the source radiographic image data; determining implantattributes, based on the source radiographic image data for the detectedimplant structure; comparing the determined implant attributes for thedetected implant structure to stored implant attributes included inimplant data records, maintained in an implant structure database, forknown manufactured implants to identify a match between the determinedimplant attributes and the stored implant attributes included in thestored implant data records; determining an outer border for thedetected implant structure appearing in the source radiographic imagedata; comparing the outer border to 2D projections maintained in aprojection dictionary, the 2D projections being at incremental distanceand angles generated from 3D dental image data, to identify a matchbetween the outer border and the 2D projections in the projectiondictionary; estimating a viewing angle at which the source radiographicimage data was obtained based on the identified match between the outerborder and the 2D projections in the projection dictionary; and derivingthe sensor pixel-to-standard-unit ratio based on the estimated angle atwhich the source radiographic image data was obtained, and based onstored geometrical information associated with a selected one of theimplant data records that most closely matches the implant attributesdetermined from the source radiographic image data.
 17. The method ofclaim 16, wherein deriving the sensor the sensor pixel-to-standard-unitratio comprises: estimating viewing angles for other dental objectsdetected in the source radiographic image data based on a position ofthe implant structure relative to the other dental structure and basedon the viewing angle at which the source radiographic image data wasobtained; and deriving the sensor pixel-to-standard-unit ratio based onthe estimated viewing angles for the other dental structures detected inthe source radiographic image data, and based on the stored geometricalinformation associated with the selected one of the implant data recordsthat most closely matches the implant attributes determined from thesource radiographic image data.
 18. A system for dental data analysiscomprising: a communication interface to obtain dental data for anindividual, the dental data comprising input radiographic image data ofat least one dental object, and treatment data representative of one ormore treatment procedures associated with the at least one dentalobject; one or more memory devices; and one or more processor-baseddevices, coupled to the communication interface and to the one or morememory devices, to: analyze, by one or more machine learning modelsimplemented by the system, the input radiographic image data to identifyone or more dental features associated with the at least one dentalobject; and derive, based on the treatment data and the identified oneor more dental features associated with the at least one dental object,one or more integrity scores for the input radiographic image data andthe treatment data, the one or more integrity scores representative ofpotential integrity problems associated with the input radiographicimage data and the treatment data, wherein the one or moreprocessor-based devices configured to derive the one or more integrityscores are configured to derive a provider score representative of apotential integrity problem associated with a dental-care providersubmitting the treatment data.
 19. The system of claim 18, wherein theone or more processor-based devices configured to analyze the inputradiographic image data are configured to: detect anomalous features inthe input radiographic image data, including determine one or more of:whether a portion of the input radiographic image data substantiallymatches a portion of a previously stored radiographic image, or whethera portion of the input radiographic image data was modified.
 20. Anon-transitory computer readable media storing a set of instructions,executable on at least one programmable device, to: obtain by acomputing system dental data for an individual, the dental datacomprising input radiographic image data of at least one dental object,and treatment data representative of one or more treatment proceduresassociated with the at least one dental object; analyze, by one or moremachine learning models implemented by the computing system, the inputradiographic image data to identify one or more dental featuresassociated with the at least one dental object; and derive, by thecomputing system, based on the treatment data and the identified one ormore dental features associated with the at least one dental object, oneor more integrity scores for the input radiographic image data and thetreatment data, the one or more integrity scores representative ofpotential integrity problems associated with the input radiographicimage data and the treatment data, wherein the instructions to derivethe one or more integrity scores comprise one or more instructions toderive a provider score representative of a potential integrity problemassociated with a dental-care provider submitting the treatment data.